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ABSTRACT 


Software  process  improvement  initiatives  are  not  unlike  other  process 
reengineering  efforts.  They  are  influenced  by  such  dynamics  as  resistance  to  change, 
organizational  structure,  cultural  barriers,  and  other  issues.  An  effective  plan  for  software 
process  improvement  implementation  must  address  concepts  of  organizational  change.  In 
this  thesis  three  perspectives  on  organizational  change  provide  the  frameworks  for 
analyzing  the  software  process  improvement  efforts  of  four  organizations.  Based  on  the 
change  theory  and  implementation  strategies  of  four  organizations  best  practices  relative 
to  preparing  an  organization  for  process  improvement,  implementing  process 
improvements,  and  sust^ing  the  improvement  effort  are  derived.  A  process  improvement 
survey,  archival  material,  personal  interviews  and  site  visits  provide  data  on  the  process 
improvement  efforts  of  the  Financial  Systems  Activity  Kansas  City.  These  data  are 
analyzed  to  characterize  the  challenges  to  the  organization’s  process  improvement  efforts. 
Recommendations  for  mitigating  these  challenges  are  provided.  The  recoimnendations 
include  an  explicit  design  for  planned  change,  transition  management  teams,  piloting, 
integration  of  process  improvement  activities  into  the  project  cycle,  and  scanning  the  . 
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I.  INTRODUCTION 


A.  BACKGROUND 

In  an  age  where  one  can  barely  keep  up  with  the  technological  pace  of  computer 
hardware,  the  consistent  production  of  quality  software  remains  a  mystery.  While 
hardware  development  and  production  tends  to  be  routine  and  predictable,  poor  quality, 
cost  and  schedule  overruns,  and  project  cancellations  plague  software  development. 

That  we  have  been  in  this  "software  crisis"  for  nearly  three  decades  hints  that 
there  are  some  real  problems  with  software  development.  The  requirement  for  quality 
software  has  increased  constantly  over  the  years.  However,  our  ability  to  produce 
software  has  remained  essentially  stagnant  (STSC,  1996).  The  state  of  software  today 
does  not  reflect  three  decades  of  improvement,  especially  when  compared  to 
improvements  in  computing  hardware  over  the  same  period. 

There  are  many  factors  that  contribute  to  the  problems  in  software  development; 
these  include  programmer  skills,  requirements  analysis,  user  expectations,  and  project 
management.  First,  the  software  development  industry  has  probably  the  widest  variation 
in  its  level  of  professional  knowledge  of  any  knowledge  discipline  (Osmundson,  1997). 
The  disparity  between  a  good  programmer  and  one  who  is  not  can  spell  disaster  for  many 
development  projects;  and  tight  markets  keep  the  best  software  engineers  in  short  supply. 

Another  factor  is  the  adequacy  of  requirements' analysis  and  planning.  The 
emphasis  on  producing  software  on  schedule  over  producing  quality  software  not  only 
has  detrimental  effects  on  software  quality  but  actually  makes  projects  late.  In  their  zeal 
to  begin  programming  as  soon  as  possible  programmers  take  shortcuts  in  requirements 
analysis  and  specifications  that  are  more  costly  in  the  later  stages  of  a  project. 
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Moreover,  many  projects  begin  behind  schedule  because  of  poor  cost  and  schedule 
estimates  which  often  result  in  understaffing  and  unattainable  deadlines. 

A  third  factor  is  customer  expectations.  Rapid  developments  in  hardware  fuel 
unrealistic  user  expectations  for  the  pace  of  advancements  in  software  (Lowry,  1991). 
These  users  also  tend  to  change  software  expectations  as  a  project  progresses,  often  times 
shifting  requirements  through  the  well  known  phenomenon  of  "requirements  creep"  even 
into  the  implementation  phase. 

Finally,  poor  management  and  the  absence  of  process  credibility  are  noted  as 
primary  contributors  to  software  failures  (STSC,  1996).  Contrary  to  popular  perceptions, 
software  engineering  professionals  have  indicated  that  software  failures  are  more  related 
to  poor  project  management  skills  than  they  are  to  the  technical  issues  of  software 
development. 

Software  development  as  a  discipline  is  not  keeping  pace  with  our  growing 
reliance  on  computer  systems.  In  the  past,  industry’s  attack  on  the  software  crisis  has 
been  focused  mainly  on  development  of  structured  software  methodologies  and 
development  of  simpler  programming  languages  and  tools  (Lowry,  1991).  More 
recently,  software  process  improvement  is  gaining  momentum  as  the  cure  for  software 
ills.  Software  process  improvement  activities  describe  those  practices  and  procedures 
aimed  at  facilitating  the  consistent  and  predictable  production  of  quality  software  (Paulk, 
1995).  Improving  software  development  processes  may  be  the  best  way  to  mitigate  the 
aforementioned  development  problems.  As  in  other  industries  mature  processes  result  in 
efficient  production  and  consistent  quality. 
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B.  SPI  INFRASTRUCTURE 


The  first  step  in  improving  software  processes  is  to  establish  a  supporting 
infrastructure.  This  infrastructure  is  the  foundation  for  the  activities,  relationships,  and 
structures  necessary  for  the  software  process  improvement  effort.  The  form  and  content 
of  a  SPI  infrastructure  are  as  varied  as  the  organizations  that  develop  them.  However, 
some  baseline  components  of  a  SPI  infrastructure  are  a  software  engineering  process 
group  (SEPG),  a  steering  committee,  process  action  teams,  and  a  process  improvement 
model. 

1.  Software  Engineering  Process  Group 

A  software  engineering  process  group  is  an  organization's  focal  point  for  process 
improvement  activities.  It  is  a  group  of  specialists  who  facilitate  the  definition, 
maintenance,  and  improvement  of  the  software  process  used  by  the  organization  (Paulk, 
p.  63).  This  group  takes  the  lead  in  implementing  and  sustaining  process  change. 
Effective  SEPGs  require  a  broad  and  diverse  mix  of  skills  including  specialized  software 
engineering  principles,  effective  communications,  negotiation,  mediation,  and 
salesmanship  to  name  a  few.  The  size  of  a  SEPG  is  usually  about  1-2%  of  the 
organization's  workforce.  Typically  the  group  is  comprised  of  high  achievers  from 
various  sections  within  the  organization. 

2.  Steering  Committee 

Direction  and  oversight  of  the  SPI  effort  is  provided  by  a  steering  committee.  The 
name  of  this  group  may  differ  from  organization  to  organization.  The  committee  provides 
the  vision  and  the  resources  to  support  the  SPI  effort.  Membership  in  the  steering 
committee  usually  includes  the  director,  assistant  director,  senior  managers,  and  the  head 
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of  the  SEPG.  One  of  the  most  important  functions  of  this  committee  is  to  communicate  a 
clear  vision  of  the  organization's  process  improvement  goals  and  support  from  the  highest 
echelons  of  the  organization. 

3.  Process  Action  Teams 

Process  action  teams  (sometimes  called  working  groups)  are  the  implementation 
arms  of  the  SPI  program.  Members  are  skilled  in  specific  disciplines  of  software 
engineering.  Their  tasks  involve  defining  and  implementing  specific  process 
improvement  activities.  Their  input  is  key  in  ensuring  that  a  proposed  change  is  realistic 
and  in  assessing  the  best  means  for  transition.  Participation  in  a  process  action  team  is 
often  on  a  rotating  or  part-time  basis.  The  size  of  the  process  action  team  varies  with  the 
organization. 

4.  Related  Software  Groups 

Related  software  groups  are  a  collection  of  individuals  t^oth  managers  and 
technical  staff)  representing  a  software  engineering  discipline  that  supports,  but  is  not 
directly  responsible,  for  performing  software  development  and/or  maintenance  (Paulk,  p. 
63).  Two  of  the  more  important  software  related  groups  are  the  software  configuration 
management  group  and  the  software  quality  assurance  group.  In  most  organizations  these 
groups  are  highly  involved  in  the  process  improvement  effort. 

5.  Capability  Maturity  Model 

There  are  a  couple  of  software  process  improvement  models,  including  the 
International  Standards  Organization  9000  series  (ISO  9000),  the  Software  Process 
Improvement  and  Capability  dEtermination  (SPICE)  and  the  Capability  Maturity  Model 
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(CMM).  This  thesis  examines  organizations  that  use  the  CMM  as  the  framework  for  its 
process  improvement  effort. 

The  Capability  Maturity  Model  is  a  framework  that  describes  the  key  elements  of 
an  effective  software  process.  It  is  a  tool  for  evaluating  an  organization’s  software 
development  process.  The  model,  based  on  industry  best  practices,  covers  all  areas  of 
software  development  including  project  planning,  requirements  analysis,  software 
metrics,  configuration  management,  and  project  management.  It  was  designed  to  help 
developers  select  process  improvement  strategies  by  determining  their  process  maturity 
and  identifying  the  most  critical  issues  to  improving  their  software  quality  and  process 
(Paulk,  p.l9).  The  model  was  created  by  the  Software  Engineering  Institute  (SEI)  a 
federally  funded  research  and  development  center  located  at  Camegie-Mellon  University 
in  Pittsburgh.  The  SEI  was  commissioned  by  the  Department  of  Defense  to  help  provide 
leadership  in  advancing  the  state  of  the  practice  of  software  engineering  and  to  reduce 
variability  and  unpredictability  in  the  software  development  process  (Paulk,  p.4).  The 
first  version  of  the  CMM  was  released  in  1991. 

The  CMM  consists  of  five  maturity  levels  that  describe  the  process  capability  of 
an  organization  from  adhoc  processes  to  a  continuous  improvement  environment.  The 
CMM  structure  consists  of  maturity  levels,  key  process  areas,  common  features,  and  key 
practices.  Figure  1.1  is  a  diagram  of  the  CMM  structure.  Each  maturity  level  comprises  a 
set  of  process  goals  that  when  satisfied  stabilize  an  important  component  of  a  software 
process  (Paulk,  p.20). 
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Indicate 


Maturity  Levels 


Activities  or 
Infrastructure 


Figure  1.1  The  CMM  structure  (Paulk,  p.  31) 

Paulk  (1995)  offers  the  following  brief  descriptions  of  the  five  maturity  levels; 


1.  Initial  The  software  process  is  characterized  as  ad  hoc,  and  occasionally 
even  chaotic.  Few  processes  are  defined,  and  success  depends  on 
individual  efforts  and  heroics. 


2.  Repeatable  Basic  project  management  processes  are  established  to  track  cost, 

schedule,  and  functionality.  The  necessary  process  discipline  is  in 
place  to  repeat  earlier  successes  on  projects  with  similar 
applications. 

3.  Defined  The  software  process  for  both  management  and  engineering 

activities  is  documented,  standardized,  and  integrated  into  a 
standard  software  process  for  the  organization.  All  projects  use  an 
approved,  tailored  version  of  the  organization’s  standard  software 
process  for  developing  and  maintaining  software. 

4.  Managed  Detailed  measures  of  the  software  process  and  product  quality  are 

collected.  Both  the  software  process  and  products  are 
quantitatively  understood  and  controlled. 
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5.  Optimizing  Continuous  process  improvement  is  enabled  by  quantitative 

feedback  from  the  process  and  from  piloting  innovative  ideas  and 
technologies. 

Key  process  areas  (KPAs)  highlight  the  issues  that  must  be  resolved  to  achieve 
each  maturity  level.  There  are  a  total  of  18  key  process  areas.  These  related  activities 
when  performed  collectively  achieve  a  set  of  goals  considered  important  for  enhancing 
process  capability.  Figure  1.2  is  a  diagram  of  the  KPAs  for  each  maturity  level. 
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describe  the  activities  and  infrastructure  that  contribute  to  the  effective  implementation 
and  institutionalization  of  a  KPA. 

The  key  practices  for  each  KPA  are  organized  by  the  following  common  features: 

•  Commitment  to  perform  typically  involves  establishing  organizational  policies 
and  leadership  to  ensure  the  establishment  and  sustainment  of  a  process. 

•  Ability  to  Perform  involves  the  resources,  organizational  structures,  and  training 
that  must  exist  to  implement  a  process. 

•  Activities  Performed  describes  the  activities,  roles,  and  procedures  necessary  to 
implement  a  key  process  area. 

•  Measurement  and  Analysis  describes  the  basic  measurement  practices  that  are 
necessary  to  determine  status  related  to  the  process. 

•  Verifying  Implementation  describes  the  steps  to  ensure  that  the  activities  are 
performed  in  compliance  with  the  process  that  has  been  established. 

These  attributes  are  indicators  of  the  effectiveness  of  the  implementation  and 
institutionalization  of  key  processes.  (Paulk,  1995)  Refer  to  figure  1.1. 

6.  Software  Process  Assessments  and  Capability  Evaluations 
Software  Process  Assessments  (SPAs)  and  Software  Capability  Evaluations 
(SCEs),  tools  developed  by  SEI,  are  key  parts  of  an  SPI  infrastructure.  SPAs  and  SCEs 
are  the  measurement  tools  of  the  CMM.  Process  assessments  are  performed  by  software 
professionals  to  determine  the  state  of  an  organization's  current  software  process.  In 
addition  to  identifying  the  key  software  process  related  issues  facing  the  organization 
they  also  generate  support  for  process  improvement  efforts.  A  process  assessment  is  a 
critical  step  in  an  internal  process  improvement  program.  (Paulk,  1995) 
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Software  Capability  Evaluations  are  performed  by  software  professionals  to 
identify  contractors  qualified  to  perform  software  work  or  monitor  the  state  of  the 
software  process  used  on  an  existing  software  effort.  An  organization’s  software  process 
capability  is  one  way  to  predict  the  likely  outcome  of  the  next  software  project  it  will 
undertake.  Capability  evaluations  are  performed  by  individuals  external  to  the 
organization  and  are  being  used  increasingly  as  a  criteria  in  source  selection  by  agencies 
with  Department  of  Defense.  (Paulk,  1995) 

The  CMM  provides  a  general  framework  for  process  improvement.  However, 
actual  methods  of  implementation  of  CMM  key  practices  are  organizationally  dependent, 
and  implementing  process  improvement  initiatives  is  challenging  and  time  consuming. 
Much  more  needs  to  be  learned  about  how  the  best  software  organizations  are  able  to 
increase  process  maturity  while  others  remain  stagnant.  The  foundations  of  process 
improvement  are  sound  management  and  a  commitment  to  continuous  improvement. 
While  these  concepts  seem  obvious  they  separate  the  few  great  organizations  from  the 
many  mediocre  and  poor  organizations. 

C.  THE  VALUE  OF  MATURITY 

It  is  important  that  leaders  understand  the  value  of  a  mature  software  process. 
Table  1.1  compares  an  immature  software  organization  with  a  mature  software 
organization. 
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Immature  Organizations 


Mature  Organizations 


Software  processes  are  improvised  by  practitioners  during  the 
course  of  the  project. 

Software  processes  are  defined  across  the  organization. 

Firefighting  mindset;  focuses  on  immediate  crises. 

Focus  on  improvement  of  processes  through  pilot  studies,  cost- 
benefit  analysis  and  other  techniques. 

Schedule  and  budgets  are  routinely  exceeded. 

Realistic  estimates  of  schedule  and  budget  based  on  historical 
performance  are  usually  achieved. 

Product  quality  is  unpredictable. 

Quantitative  methods  exist  forjudging  product  quality. 

Table  1.1  Immature  versus  mature  software  organizations 


Software  process  improvement  is  an  expensive,  complicated,  and  time-consuming 
undertaking.  The  SEI  maintains  a  database  of  the  maturity  levels  of  organizations 
reporting  CMM-based  process  improvement  efforts.  Figure  1.3  shows  the  organization 
maturity  profile  as  of  May  1^98.  The  profile  is  compiled  by  SEI  and  characterizes  the 
software  process  maturity  of  the  software  community.  The  profile  is  based  on  input  from 
700  software  organizations  representing  commercial  industry,  DoD  contractors,  military 
and  other  contractors  worldwide.  The  profile  indicates  the  relatively  immature  state  of 
software.  These  figures  were  compiled  from  a  variety  of  organizations.  At  this  time 
less  than  one  percent  of  the  assessed  organizations  are  CMM  Level  5  organizations. 
Process  improvement  issues  and  strategies  from  two  CMM  Level  5  organizations  are 
reviewed  in  this  study. 
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Organization  Maturity  Profile 

May  1998 


Figure  1.3  Organization  Maturity  Profile  (Source  SEI) 

D.  BENEFITS  OF  CMM-BASED  PROCESS  IMPROVEMENT 

CMM-based  process  improvement  is  but  one  strategy  that  is  available  for  an 
organization  to  improve  its  performance.  Quantifying  the  return  on  investment  of  CMM- 
based  process  improvement  is  a  contentious  issue  among  industry  leaders.  The 
parameters  forjudging  its  effectiveness  are  many  times  more  qualitative  than 
quantitative.  However,  in  a  technical  report  published  by  the  SEI  (SEI/CMM  TR-13)  data 
from  a  diverse  group  of  13  organizations  (representing  DoD  contractors,  military  and 
commercial  software  organizations)  were  analyzed  to  assess  the  benefits  of  CMM-based 
process  improvement.  The  report  indicates  significant  gains  in  productivity,  cycle  time, 
and  product  quality  from  organizations  using  CMM-based  process  improvement. 
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E.  CHALLENGES  OF  SOFTWARE  PROCESS  IMPROVEMENT 

The  challenges  to  process  improvement  are  many.  In  the  schedule  driven  world  of 
software  development,  poorly  planned  process  improvement  efforts  are  quickly 
overwhelmed  by  delivery  dates.  Improved  processes  do  not  represent  the  tangible  outputs 
(i.e.,  a  completed  project)  that  organizations  are  accustomed  to.  The  connection  between 
the  strategic  goals  of  the  organization  and  process  improvement  efforts  can  be  vague  or 
nonexistent  to  employees.  Process  improvement  is  not  easy.  It  requires  training  on  the 
part  of  all  parties  associated  with  the  development  process.  Sluggish  starts  and  initial 
mistakes  that  are  common  in  most  software  projects  contribute  to  organizational 
resistance  when  they  are  encountered  during  the  process  improvement  effort.  Software 
developers  of  the  "programming  is  art"  mindset  view  attempts  to  manage  the 
development  process  as  a  hindrance.  Efforts  at  creating  an  environment  of  continuous 
process  improvement  must  overcome  the  security  that  comes  from  familiar  routines. 

F.  CASE  STUDIES 

Because  of  the  time,  expense  and  complexity  involved  many  organizations  have 
abandoned  their  process  improvement  efforts  (after  incurring  great  costs).  However, 
several  case  studies  document  well  conceived  and  implemented  process  improvement 
efforts.  By  examining  the  experiences  of  different  types  of  software  organizations  and 
their  approaches  to  software  process  improvement,  a  set  of  generalizable  best  practices 
may  be  derived. 

Software  process  improvement  initiatives  are  not  unlike  other  process  reengineering 
efforts.  They  are  influenced  by  such  dynamics  as  resistance  to  change,  organizational 
stmcture,  cultural  barriers,  and  other  issues.  Software  engineers  must  understand  the  need 
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to  change,  be  convinced  the  new  process  will,  indeed,  improve  performance,  and  be 
supported  as  they  learn  and  implement  it.  (STSC,  1996)  An  effective  plan  for  software 
process  improvement  implementation  must  address  concepts  of  organizational  change 
including  sociotechnical  change,  transition  management,  cultural  barriers,  organizational 
structure,  and  resistance  to  change. 

G.  STUDY  PURPOSE 

The  purpose  of  the  study  is  to  develop  ways  to  facilitate  implementation  of 
software  process  improvement  initiatives.  Initially  change  management  theory  will  be 
examined  to  extract  those  themes  relevant  to  successful  implementation  and  sustainment 
of  change.  Then  the  software  process  improvement  initiatives  of  a  broad  cross  section  of 
organizations  are  reviewed  in  the  context  of  relevant  change  theory.  By  analyzing  various 
implementation  techniques  and  the  associated  results,  best  practices  will  be  derived. 

This  research  targets  the  Defense  Finance  and  Accounting  Service  (DFAS), 
Financial  Systems  Agency,  Kansas  City  (FS A-KC)  which  provides  resources  and  support 
in  performing  a  variety  of  software  development  services  for  the  U.  S.  Marine  Corps  and 
other  Department  of  Defense  (DoD)  organizations.  FSA-KC  is  in  the  midst  of  a  software 
process  improvement  effort  that  began  in  1993.  Information  gathered  from  on-site 
analysis  of  FS  A's  current  software  process  improvement  efforts,  interviews  with  FS  A 
personnel,  archival  material,  and  a  survey  provide  data' for  this  thesis. 

The  objective  of  this  thesis  is  to  apply  software  process  improvement  best 
practices  and  change  management  theory  to  the  Financial  Services  Agency  (FSA)  of  the 
Defense  Finance  and  Accounting  Service  (DFAS)  in  an  attempt  to  facilitate  their  ongoing 
process  improvement  efforts. 
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H.  RESEARCH  QUESTIONS 

Primary: 

•  How  can  the  software  improvement  process  be  facilitated  to  increase  efficiency? 
Secondary: 

•  What  is  the  software  process  improvement  process?  What  problems  does  it  face? 

•  What  are  software  process  improvement  best  practices? 

•  How  can  we  apply  change  management  theory  to  interpret  software  process 
improvement  best  practices? 

•  What  general  guidelines  can  be  developed  to  facilitate  the  improvement  process? 

•  What  are  the  challenges  to  software  process  improvement  efforts  at  FS A? 

•  How  can  these  challenges  be  mitigated  using  software  process  improvement  best 
practices  and  change  management  concepts? 

L  METHODOLOGY 

The  methodology  used  in  this  thesis  research  consists  of  the  following  steps: 

1.  Conduct  a  literature  search  of  journals,  books,  magazine  articles,  web 
resources,  and  other  library  information  resources  on  change  management 
concepts. 

2.  Conduct  a  review  of  related  change  management  theory. 

3.  Conduct  a  review  of  software  process  improvement  case  studies. 

4.  Compile  a  list  of  software  process  improvement  best  practices  based  on 
common  themes  from  successful  cases. 
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5.  Design  and  conduct  semi-structured  interviews  for  assessment  of  FS  A-KC 
focusing  on  facts  known  to  influence  the  effectiveness  of  improvement  efforts 
based  on  data  from  case  studies,  consultant  interviews,  and  literature  reviews. 

6.  Evaluate  interview  results  and  characterize  FSA-KC  environment. 

7.  Prepare  recommendations  for  implementation  of  software  process 
improvement  initiatives  at  FSA-KC. 

J.  SIGNIFICANCE  OF  THE  STUDY 

This  study  provides  recommendations  to  FSA-KC  for  successful  implementation 
and  maintenance  of  software  process  improvement  initiatives.  It  serves  as  an  example  for 
other  organizations  seeking  to  implement  software  process  improvement  initiatives. 
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II.  REVIEW  OF  RELATED  LITERATURE 

A.  INTRODUCTION 

Software  process  improvement  initiatives  are  not  unlike  other  process 
reengineering  efforts.  They  are  influenced  by  such  dynamics  as  resistance  to  change, 
organizational  structures,  and  cultural  barriers  to  name  a  few.  A  review  of  related  change 
management  theory  provides  a  context  for  examining  SPI  implementation  issues  within 
these  constraints.  Specifically,  three  theoretical  perspectives  on  organizational  change 
provide  the  frameworks  for  analysis  in  this  study. 

Dalziel  and  Schoonover  (1988)  examine  the  problems  of  implementing 
organizational  change  and  provide  recommendations  for  mitigating  these  problems. 
Pasmore  (1988)  approaches  the  problems  of  change  implementation  from  the  standpoint 
of  optiihization  of  the  organization's  social  and  technical  systems  within  the  constraints 
of  the  environment  it  operates  in.  Finally,  Bridges  (1991)  espouses  the  importance  of 
managing  transitions— the  ending,  the  neutral  zone,  and  the  beginning-as  the  key  to 
implementing  and  sustaining  change.  These  three  perspectives  are  reviewed  below.  This 
section  concludes  with  a  summary  of  the  key  themes. 

B.  CHANGING  WAYS 

In  "Changing  Ways",  Dalziel  and  Schoonover  (1988),  suggest  that  effective 
change  leaders  deal  with  the  tangible  and  hidden  processes  of  change  by  answering  three 
questions: 

•  Have  we  got  our  organization  ready  for  change? 

•  Do  we  have  the  right  mix  of  skills  on  our  team  to  make  the  change  happen? 
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•  Can  we  ensure  that  the  implementation  process  will  be  successful?  (Dalziel 
and  Schoonover,  p.l5) 

1.  Getting  the  Organization  Ready  for  Change 

Dalziel  and  Schoonover  (1988)  offer  the  following  as  attributes  of  organizational 
readiness: 

•  History  of  Change:  The  prior  experience  of  the  organization  in  accepting 
change. 

•  Clarity  of  Expectations:  The  degree  to  which  the  expected  results  of  change 
are  shared  across  various  levels  of  the  organization. 

•  Origin  of  the  Idea  or  Problem:  The  degree  to  which  those  most  affected  by  the 
change  initiated  the  idea  or  problem  the  change  solves. 

•  Support  of  Top  Management:  The  degree  to  which  top  management  sponsors 
the  change.  • 

•  Compatibility  of  Organizational  Goals:  The  degree  to  which  the  proposed 
change  corresponds  to  past  and  present  organizational  practices  and  plans. 


The  first  three  dimensions— history  of  change,  clarity  of  expectations,  and  origin 
of  the  request  or  problem— focus  on  how  to  motivate  people  to  embrace  change.  To 
stimulate  a  positive  attitude  toward  change,  change  leaders  let  "success  breed  success", 
by  scaling  initial  interventions  to  their  organization's  past  performance  (Dalziel  and 
Schoonover,  p.l5).  The  most  important  thing  is  not  to  fail  at  the  start.  You  should  always 
ensure  an  initial  success-even  if  you  have  to  move  more  slowly  or  only  start  a  small  part 
of  the  change.  In  addition  change  leaders  instill  consistent  expectations  about  the  change 
and  its  ramifications  throughout  the  organization.  Finally,  they  make  sure  that  change  is 
framed  to  meet  key  problems  of  those  who  will  have  to  live  with  the  change  first. 

(Dalziel  and  Schoonover,  1988) 

Support  of  top  management  is  critical  at  the  initial  stages  of  planned  change  and 
important  throughout  the  process.  Upper-level  alliances  provide  tangible  support  in  the 


18 


form  of  resources  for  various  phases  of  the  project  and  intangible  support  in  the  form  of 
sponsorship  and  networking.  Change  managers  managing  successful  endeavors  generate 
support  much  more  often  than  those  producing  less  successful  results.  Dalziel  and 
Schoonover  (1988)  suggest  building  a  case  for  change  that  appeals  to  top-level  concerns 
and  creating  a  formal  management  review  process  involving  key  players  in  top- 
management  as  ways  that  change  leaders  can  build  top-management  support. 

The  best  change  leaders  tailor  the  scope,  scale,  and  type  of  change  to  fit  existing 
patterns.  This  keeps  the  effort  practical  (Dalziel  and  Schoonover,  pl7).  A  change  that 
aligns  with  existing  goals,  practices,  and  priorities  of  the  organization  is  more  easily 
understood  and  adopted.  Every  leader  who  initiates  a  policy,  program,  or  practice  should 
assess  the  compatibility  of  the  change  with  the  current  organizational  culture; 

•  Does  the  change  fit  with  an  overall  business  plan? 

•  Do  proposed  changes  make  employees’  jobs  harder  or  easier? 

•  Is  the  change  technically  familiar  to  members  of  the  organization? 

Dalziel  and  Schoonover  (1988)  suggest  that  change  agents  integrate  the  change 

into  on-going  procedures  whenever  possible.  Another  technique  is  to  implement  the  . 
change  initially  in  the  most  accepting  surroundings  (i.e.,  a  section  with  particularly 
knowledgeable  and  flexible  workers).  They  also  highlight  the  importance  of 
communicating  how  the  change  fits  with  organizational  directions. 

2.  Implementing  the  Change 

As  the  change  effort  evolves,  the  change  leader  must  address  a  number  of  key 
issues  to  translate  proposed  actions  into  on-going  organizational  practices.  A  set  of  five 
key  processes  support  the  process  of  successful  implementations: 


19 


a.  Clarifying  Plans 

Clarifying  plans  is  the  process  in  which  implementors  define,  document, 
and  specify  the  change.  Dalziel  and  Schoonover  (1988)  emphasize  that  successful  change 
leaders  make  plans  public  and  involve  influential  members  of  the  end-user  group  to 
participate  in  formulating  the  plan.  They  conduct  an  on-going  dialogue  about  each  of  the 
steps  in  their  plan,  with  both  the  change  agents  and  the  end-users. 

b.  Integrating  New  Practices 

Integrating  new  practices  is  the  process  in  which  an  organization 
incorporates  change  into  its  operations.  Change  managers  should  gradually  integrate 
change  efforts  into  an  organization,  gearing  the  rate  of  change  to  the  organizational 
context  rather  than  cramming  it  into  a  prefixed  timeline.  Limiting  the  rate  of  change  until 
acceptance  and  understanding  is  clear  produces  more  effective  long-term  results.  It  is  also 
important  to  limit  the  amount  of  change  introduced  at  one  time. 

c.  Providing  Education 

Education  provides  programs  in  which  end  users  leam  about  and  use  new 
approaches  and  procedures.  A  good  training  program  is  designed  from  the  user’s 
perspective.  It  incorporates  their  knowledge  and  experience,  and  is  constantly  evaluated 
in  the  context  of  its  affect  on  work  practices  and  user  attitudes. 

d.  Fostering  Ownership 

Fostering  ownership  is  the  process  through  which  end  users  come  to 
identify  new  processes  and  procedures  as  their  own,  rather  than  regarding  them  as 
changes  imposed  upon  them.  Ownership  is  fostered  when  the  primary  benefits  of  the 
change  are  apparent  to  the  end-users  and  when  the  end-users  are  involved  in  the  plans. 
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decisions  and  outcomes  of  the  effort.  When  users  understand  and  participate,  they 
become  committed  to  current  and  future  strategies. 

e.  Feedback 

The  feedback  process  is  accomplished  by  first  documenting  and 
describing  the  expected  outcomes  of  the  change,  and  then  using  input  from  those  affected 
by  the  change  to  judge  the  effectiveness  of  the  change  during  and  immediately  after 
implementation.  Effective  change  managers  perform  significant  up-front  information 
gathering  and  continue  exploring  options  throughout  the  change  effort.  Assessing 
employee  and  management  attitudes  and  practices  is  key  to  successful  change 
implementation.  The  broad  perspective  gained  from  multiple  inputs  that  include  various 
business  functions  and  customers  often  generate  ideas  for  more  practical  effective  change 
efforts.  (Dalziel  and  Schoonover,  1988)  , 

C.  THE  SOCIOTECHNICAL  SYSTEMS  PERSPECTIVE 

Pasmore  (1988)  asserts  that  an  organizations  effectiveness  is  a  function  of  the 
people  (the  social  system),  the  tools  and  techniques  they  use  (the  technical  systems)  and 
the  customers  (the  environment).  By  optimizing  the  social  and  technical  elements  of 
organizational  design  within  the  context  of  the  environment,  organizations  may  achieve 
peak  performance  and  efficiency.  Conversely,  failures  in  organizational  undertakings  can 
be  traced  to  overlooking  the  significance  of  one  or  more  of  these  elemefits.  While  it  may 
be  argued  that  there  are  other  factors  that  contribute  to  organizational  effectiveness,  it 
seems  clear  that  at  a  minimum  these  three  must  be  considered.  Pasmore  asserts  that  every 
organization  is  a  sociotechnical  system. 
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1.  The  Environment 


Every  organization  is  ultimately  dependent  on  the  environment  (in  the  form  of 
customers,  competitors,  sources  of  resources,  and  other  things)  for  survival.  The 
environment  imposes  constraints  and  opportunities  that  influence  the  goals,  processes, 
and  outcomes  of  organizational  systems.  Internal  measures  of  success  that  are  not 
consistent  with  environmental  constraints  and  opportunities  are  inaccurate  predictors  of 
an  organizations  future  success.  The  external  environment  is  the  ultimate  judge  of  system 
effectiveness.  The  sociotechnical  systems  perspective  asserts  that  whatever  decisions  are 
made  about  organizational  design  should  meet  the  demands  of  the  external  environment 
as  well  as  the  internal  social  and  technical  systems.  (Pasmore,  1998) 

As  a  basic  tenet,  organizational  designs  should  "fit"  with  the  environment  (Kotter, 
1978)  but  rapid  and  unpredictable  changes  in  the  environment  make  achieving  the  proper 
fit  difficult.  To  achieve  and  maintain  an  optimal  fit  between  the  organization  and  the 
environment,  the  organization  must  also  change  continuously.  Therefore  managers 
should  strive  to  create  a  learning  capacity  within  the  organization,  to  facilitate  continuous 
improvement  of  organizational  practices  and  systems.  (Pasmore,  1988) 

Pasmore  asserts  that  the  greater  the  level  of  experimentation  in  organization 
design,  the  greater  the  likelihood  that  learning  will  occur  and  lead  to  more  effective 
future  adaptations  to  the  environment.  However,  he  realizes  that  opportunities  for 
managers  to  experiment  with  different  techniques  that  enhance  their  learning  is  difficult 
in  the  face  of  pressures  to  settle  upon  a  workable  solution  that  can  deal  with  immediate 
environmental  demands.  (Pasmore,  1988) 
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The  environment  can  be  seen  as  a  provocation  or  a  source  of  inspiration 
(Pasmore,  p21).  The  more  the  environment  is  viewed  as  a  source  of  provocation,  the 
more  adaptation  will  focus  on  solving  immediate  problems  versus  innovations  in 
organizational  design.  Companies  that  are  successful  in  turbulent  environments  do  more 
than  react  to  competition;  they  take  steps  to  transform  the  environment  itself  to  make  it 
more  conducive  to  their  well  being.  (Pasmore,  1988) 

Given  the  impact  of  the  environment  on  organizational  success,  it  is  clear  that 
sociotechnical  systems  design  must  begin  with  an  appreciation  of  organization- 
environment  relationships.  Pasmore  (1988)  details  two  methods  for  scanning  the 
environment;  open  systems  planning  and  the  search  conference.  The  methods  are  very 
similar.  Both  methods  assess  the  organization-environment  fit,  within  the  context  of  a 
particular  change  initiative.  Table  2.1  illustrates  the  general  approach  to  the  two  scanning 
methods. 


Open  Systems  Planning 

Search  Conference 

List  the  important  stakeholders  in  the  environment  and 
expectations  they  hold  regarding  both  how  the  organization  will 
operate  and  what  it  will  achieve. 

The  external  social  field  (contextual  environment)  is  explored. 

Create  a  realistic  future  scenario  which  depicts  what  would 
happen  if  it  continued  on  it’s  current  course. 

Discuss  broad  desirable  futures  for  the  organization. 

Create  an  idealistic  future  scenario. 

Explore  the  unique  characteristics  of  the  system. 

Compare  the  realistic  and  idealistic  scenarios  to  identify  design 
constraints  and  opportunities. 

Explore  possibilities  and  constraints  of  the  system. 

Plan  specific  actions  to  support  movement  toward  the  idealistic 
future  scenario. 

Begin  operational  planning  to  achieve  the  desired  future  system. 

Table  2.1  Environmfental  Scanning  Methods 


These  techniques  for  scanning  the  environment  can  be  carried  out  at  different 
levels  of  the  organization.  Environmental  awareness  is  enhanced  by  regularly  involving 
employees  at  all  levels  in  scanning  the  environment.  Regardless  of  the  mechanisms 
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employed  there  is  greater  potential  for  generating  workable  action  plans  when  key 
external  stakeholders  are  involved  in  scanning  the  environment.  (Pasmore,  1988) 

2.  The  Social  System 

Organizational  success  depends  on  social  system  dynamics.  Therefore,  the  social 
system  should  receive  as  much  attention  during  sociotechnical  systems  design  as  the 
technology  and  the  environment  (Pasmore,  1988).  The  social  system  of  an  organization  is 
comprised  of  the  people  who  work  in  the  organization  and  all  that  is  human  about  their 
presence.  This  includes  individual  attitudes  and  beliefs,  the  implicit  psychological 
contracts  between  employees  and  employers,  reactions  to  work  arrangements  and 
company  policies,  and  a  myriad  of  other  human  dimensions.  Pasmore  (1988)  points  out 
that  analyzing  the  organization  as  a  social  system  is  difficult,  but  social  systems  are  the 
only  parts  of  the  organization  that  can  conceive  and  implement  improvements  in 
organizational  processes. 

While  encouraging  managers  to  strive  for  joint  optimization  of  social  and 
technical  systems,  Pasmore  (1988)  concedes  that  actual  optimization  of  social  and 
technical  systems  is  extremely  challenging  considering  the  complexity  and  rapidly 
evolving  nature  of  social  systems.  However,  he  points  out  the  following  predictable 
aspects  of  social  systems: 

•  The  better  the  fit  between  the  organization’s  culture  and  its  external 
environment,  the  more  effective  the  organization  will  be. 

•  The  greater  the  disparity  between  organizational  design  features  and  the 
unique  characteristics  of  organizational  members,  the  less  successful  a  design 
will  be. 
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•  The  greater  the  involvement  of  employees  in  the  design  process,  the  more 
flexible  the  resulting  organizational  design  is  likely  to  be. 

•  The  greater  the  involvement  of  employees  in  the  design  process,  the  clearer 
the  understanding  of  how  behaviors  are  linked  to  desired  rewards. 

Pasmore  (1988)  asserts  that  at  the  macro  level,  social  system  dynamics  can  be 
understood  both  in  terms  of  the  culture  of  the  organization  and  its  structure,  which 
includes  the  designation  of  departmental  boundaries,  reward  systems,  supervisory  and 
control  systems,  job  design  principles,  performance  expectations,  employee  involvement 
opportunities,  and  the  nature  of  contracts  between  labor  and  management  as  well  as 
psychological  contracts  between  the  organization  and  its  members.  The  challenge  for  the 
sociotechnical  systems  designers  is  to  first  understand  the  constraints  that  social 
djmamics  within  an  organizatipn  place  on  design  possibilities,  and  second,  to  recognize 
that  bold  design  changes  can  produce  both  desirable  and  undesirable  effects  on  behavior. 
(Pasmore,  1988) 

The  social  system  is  the  source  of  all  innovation  and  adaptation.  Because 
innovation  and  adaptation  require  human  commitment,  we  need  to  understand  how  the 
design  of  organizations  influences  the  willingness  of  people  to  help  organizations 
succeed.  High  performance  requires  commitment;  building  commitment  requires  treating 
people  like  adults  and  engaging  them  fully  in  shaping  their  future.  (Pasmore,  1988) 

3.  The  Technical  System 

Pasmore  defines  the  technical  system  of  an  organization  as  the  tools,  techniques, 
devices,  artifacts,  methods,  configurations,  procedures  and  knowledge  used  by 
organizational  members  to  acquire  inputs,  transform  inputs  into  outputs  and  provide 
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outputs  or  services  to  clients  or  customers.  The  primary  function  of  technology  is  to 
enhance  the  amount  of  work  an  individual  can  accomplish  and  the  reliability  of 
individual  performance.  (Pasmore,  1988) 

The  nature  of  the  technical  system  used  by  an  organization  influences  the 
apparent  level  of  commitment  and  motivation  demonstrated  by  individuals.  The  interplay 
between  the  roles  defined  by  technology  and  one's  self  image  gives  rise  to  psychological 
contracts  between  the  individual  and  organization  which  define  the  level  of  effort  and 
commitment  one  will  demonstrate  in  pursuing  organizational  goals.  Technologies  which 
prescribe  narrow  roles  for  individuals,  produce  psychological  contracts  which  preclude 
learning  or  change,  and  are  reinforced  by  structural  arrangements  which  interfere  with 
cooperative  problem  solving  may  result  in  total  immobilization.  Under  such 
circumstances  changes  of  almost  any  kind  are  threatening,  since  they  may  undercut  the 
personal  or  political  security  associated  with  traditional  arrangements  (Pasmore  p.62). 
The  design  of  jobs  will  be  more  stimulating  when  the  technology  is  designed  to  provide 
more  direct  and  immediate  feedback  and  when  the  technology  leaves  a  significant  degree 
of  relevant  decision  making  to  employees. 

4.  Sociotechnical  Systems  Redesign 

The  sociotechnical  systems  redesign  model  (Figure  2.1)  is  targeted  at 
implementing  changes  to  existing  systems.  This  model  for  redesign  is  consistent  with 
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Figure  2.1  Sociotechnical  Systems  Change  Model  (Adapted  from  Pasmore  p.l  1 1) 


Define  scope 


those  concepts  put  forth  in  the  sociotechnical  systems  perspective.  Pasmore  offers 

the  following  cautions  on  the  use  of  the  sociotechnical  systems  redesign  model: 

It  is  not  a  quick-fix,  limited  scope,  cosmetic  process  which  is  tightly  controlled  by 
management  alone.  If  followed  as  outlined  here,  the  model  may  take  anywhere 
from  six  months  to  three  years  to  complete;  it  will  probably  demand  full-time 
attention  by  several  organizational  members,  require  a  substantial  outlay  of 
resources,  and  call  into  question  every  aspect  of  the  organization's  design  and 
operating  practices;  moreover,  it  is  almost  certain  to  disrupt  organizational 
performance  for  a  period  of  time.  But  in  the  end,  it  is  more  likely  to  produce 
changes  that  are  desired— and  often  demanded— in  a  competitive  environment 
(Pasmore,  p.  110). 


Each  of  the  items  in  the  sociotechnical  systems  redesign  process  is  briefly 
described  below: 

Step  1:  Define  the  scope  of  the  system  to  be  redesigned. 

The  first  step  in  the  sociotechnical  systems  change  process  is  intended  to  clarify 
the  boundaries  of  the  organizational  unit  to  be  redesigned.  In  addition,  it  includes  the 
activities  associated  with  the  entry,  scouting  and  contracting  phases  in  typical 
organization  development  efforts.  These  include  defining  the  need  for  change, 
determining  the  potential  for  success,  agreeing  on  a  change  mode,  defining  rough  time 
and  cost  parameters,  clarifying  expectations  among  parties,  forming  a  steering  committee  • 
to  oversee  the  effort,  and  agreeing  to  a  public  contract  which  outlines  the  work  to  be 
done. 

Step  2:  Determine  environmental  demands. 

The  second  step  in  the  change  process  involves  identifying  importa.nt 
constituencies  in  the  external  environment  who  might  impact  the  nature  and  scope  of 
changes  that  will  occur  in  the  organization.  Bzised  on  the  perceived  demands  of 
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competitors,  shareholders,  corporate  management  and  others,  areas  of  opportunity  as  well 
as  constraints  on  the  redesign  are  identified. 

Step  3:  Create  the  vision  statement  and  charter. 

Based  on  the  environmental  demands  identified  in  Step  2,  top  level  decision 
makers  draft  a  preliminary  statement  outlining  their  vision  of  the  ideal  organization  they 
hope  to  create  through  the  change  process.  In  addition,  the  statement  outlines  the 
perceived  constraints  in  such  areas  a  policies,  reward  systems  and  labor  agreements  and 
sets  clear  goals  for  performance  after  the  redesign. 

Step  4:  Educate  organizational  members. 

Although  designated  as  a  distinct  step  in  the  change  process,  the  education  of 
organizational  members  which  begins  in  Step  4  continues  throughout  the  remainder  of 
the  change  process  and  beyond.  In  additiori  to  education  about  the  sociotechnical  systems 
perspective  and  change  process,  organizational  members  should  also,  begin  training  that 
will  prepare  them  to  assume  their  new  roles  in  the  redesigned  system. 

Step  5:  Create  the  change  structure. 

In  step  5,  a  representative  design  team  is  formed  which  will  conduct  the  actual 
sociotechnical  systems  analysis  and  formulate  proposals  for  changes  in  the  system.  The 
relationship  between  the  design  team  and  the  top  level  decision  makers  on  the  steering 
committee  is  clarified. 

Step  6:  Conduct  the  sociotechnical  systems  analysis. 

Step  6  actually  consists  of  three  distinct  but  interdependent  analyses  of  the  social 
system,  technical  system,  and  environment  of  the  organization  as  detailed  in  the  previous 
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sections.  A  clear  understanding  of  how  the  organization  currently  operates  points  toward 
areas  for  improvement  in  the  future. 

Step  7:  Formulate  redesign  proposals. 

Ideas  for  redesigning  the  organization  flow  from  the  analyses  conducted  in  Step  6 
as  well  as  the  vision  put  forth  by  the  steering  conunittee  and  revised  by  organizational 
members.  All  proposals  are  reviewed  by  organizational  members  and  checked  against 
sociotechnical  systems  principles. 

Step  8:  Implement  recommended  changes. 

In  Step  8  a  plan  is  devised  for  implementing  the  changes  which  pass  the  review  in 
Step  7.  Responsibility  for  seeing  that  the  changes  occur  is  designated  to  individuals  with 
the  power  to  make  them  happen.  A  review  system  is  created  to  monitor  implementation 
success.- 

Step  9:  Evaluate  changes  and  redesign  as  necessary. 

Since  the  process  of  sociotechnical  systems  change  is  complex,  it  is  necessary  to 
evaluate  the  changes  that  are  made  to  ascertain  whether  they  are  producing  their  intended 
effects. 

D.  MANAGING  TRANSITIONS 

The  leader  is  no  more  or  less  than  a  manager  of  transitions  (Pasmore,  p.  148). 
Bridges  (1991)  distinguishes  change  management  theory  as  focusing  on  a  desired 
outcome  that  the  change  will  produce  while  transition  management  focuses  on  the 
psychological  process  people  will  go  through  to  come  to  terms  with  the  new  situation. 
Bridges  terms  the  three  phases  of  transition  as  the  ending,  the  neutral  zone,  and  the  new 
beginning  respectively.  This  is  not  to  imply  that  the  phases  are  distinct  and  sequential,  but 
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more  to  isolate  the  dynamics  of  transition  into  its  simplest  forms.  As  depicted  by  the 
overlapping  phases  in  Figure  2.2  the  organization  will  be  in  more  than  one  phase  at  a 
time.  The  movement  through  transition  is  marked  by  a  change  in  the  dominance  of  one 
phase  as  it  gives  way  to  the  next.  (Bridges,  1991) 


Figure  2.2  The  Three  Phases  of  Transition  (Adapted  from  Bridges  p.  70) 

Transition  management  recognizes  the  fact  that  one  of  the  major  challeng6s  of 
implementing  and  sustaining  change  is  getting  people  to  stop  doing  the  things  that  they 
have  grown  accustomed  to  doing.  It  also  recognizes  the  significance  of  that  time  between 
the  ending  of  established  practices  and  the  beginning  of  new  practices.  This  is  a  time  of 
increased  organizational  vulnerability,  but  can  also  be  a  time  of  increased  creativity  and 
ingenuity.  Unless  transition  occurs,  change  will  not  work.  (Bridges,  1991) 

1.  The  Ending 

While  the  first  task  of  change  management  is  to  understand  the  destination  and 
how  to  get  there,  the  first  task  of  transition  management  is  to  convince  people  to  leave 
home  (Bridges,  p.32).  The  starting  point  for  transition  is  not  the  outcome  but  the  ending 
that  you  will  have  to  make  to  leave  the  old  situation  behind.  If  things  change  at  least 
some  employees  and  managers  are  going  to  have  to  let  something  go.  So  beginnings 
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depend  on  endings.  Bridges  (1991)  cites  the  connection  between  resistance  to  change  and 
the  losses  associated  with  endings.  It  isn't  the  changes  themselves  that  people  in  these 
cases  resist.  It’s  the  losses  and  endings  that  they  experience  and  the  transition  that  they 
are  resisting.  The  failure  to  identify  and  be  ready  for  the  endings  and  losses  that  change 
produces  is  the  largest  single  problem  that  organizations  in  transition  encounter. 

Bridges  (1991)  recommends  providing  frequent  and  continuous  communication  as 
a  solution  for  managing  the  impact  of  the  ending  on  people.  Change  agents  should 
describe  the  change  in  as  much  detail  as  possible.  They  should  present  the  innovations  as 
developments  that  build  on  the  past  and  help  realize  its  potential. 

2.  The  Neutral  Zone 

The  neutral  zone  is  the  no-mans  land  between  the  old  reality  and  the  new.  It  is  a 
time  when  the  old  way  is  gone  and  the  new  one  doesn't  feel  comfortable  yet.  The  neutral 
zone  is  usually  a  time  of  confusion  and  frustration,  but  if  managed  properly  can  be  a  time 
of  increased  creativity  and  innovation.  The  more  drastic  the  change  the  longer  the  time 
spent  in  the  neutral  zone  and  the  more  important  it  is  to  effectively  manage  this  time. 
(Bridges,  1991) 

To  minimize  the  confusion  and  frustration  in  the  neutral  zone  senior  management 
must  protect  their  people  from  unrelated  or  unnecessary  changes  as  much  as  possible.  If 
this  is  not  possible,  management  should  try  to  cluster  and  relate  new  changes  to  the 
primary  on-going  changes.  People  can  deal  with  a  lot  of  change  if  it  is  coherent  and  part 
of  a  larger  whole.  But  unrelated  and  unexpected  changes,  even  if  they  are  small  can  be 
the  proverbial  straw  that  breaks  the  camel's  back.  (Bridges  1991) 
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One  of  the  persistent  problems  during  transition  is  for  decision  makers  and  those 
implementing  decisions  to  be  clear  on  precisely  what  impact  the  decisions  and  actions 
will  have.  A  transition  monitoring  team  (TMT)  can  provide  feedback  and  facilitate 
communication  between  planners  and  implementors.  The  TMT's  purpose  includes: 

•  Demonstrating  organizational  concern  for  the  impact  of  transition  on  the 
people. 

•  Reviewing  plans  or  communications  before  they  are  announced. 

•  Providing  ready  access  to  the  organization's  grapevine.  Countering  rumors 
and  correcting  misinformation.  (Bridges,  p.42) 

In  the  neutral  zone  restraints  on  innovation  are  the  weakest.  Only  when  the  old 
way  of  seeing  things  disappears  are  habit  patterns  broken,  and  a  new  way  will  emerge. 
.Innovation  will  take  place  automatically  in  the  neutral  zone  if  people  are  protected  from 
further  change,  informed,  and  encouraged  to  find  new  ways  to  do  things.  (Bridges,  1991) 

3.  The  New  Beginning 

A  new  beginning  can  only  take  place  after  a  well  managed  ending  and  neutral 
zone.  Starts  involve  new  situations,  a  beginning  involves  the  new  understandings,  values, 
and  attitudes  that  are  required  for  successful  transition.  To  make  a  new  beginning  people 
need  The  Four  Ps:  the  purpose,  a  picture,  the  plan,  and  a  part  to  play.  (Bridges,  1991) 
a.  The  Purpose 

Successful  new  beginnings  are  based  on  a  clear  purpose  that  aligns  with 
the  internal  characteristics  of  the  organization  and  the  external  forces  that  influence  the 
organization.  You  need  to  explain  the  purpose  behind  the  new  beginning  clearly.  You 


33 


may  discover  that  people  do  not  have  a  realistic  idea  of  where  the  organization  really 
stood  and  what  its  problems  were  (Bridges,  p.  53). 

b.  A  Picture 

Most  of  the  pain  of  the  neutral  zone  comes  from  the  fact  that  it  is  a  time 
without  a  viable  organizational  picture.  People  in  the  organization  need  to  be  able  to 
visualize  the  change.  They  need  something  they  can  see,  at  least  in  their  imaginations. 
They  need  a  picture  of  how  the  outcome  will  look  and  they  need  to  be  able  to  imagine 
how  it  will  feel  to  be  a  participant  in  it.  (Bridges,  1991) 

c.  The  Plan 

A  transition  management  plan  differs  from  a  change  management  plan  in 
that  it  starts  with  where  the  people  are  and  works  forward  through  the  process  of  ending, 
getting  through  the  neutral  zone,  and  finally  the  new  beginning.  The  transition  plan  starts 
with  where  people  are  and  spells  out  the  details  of  the  route  form  the  current  state  to  the 
future  state.  Plans  are  immensely  reassuring  to  most  people,  not  just  because  they  contain 
information  but  because  they  exist.  (Bridges,  1991) 

d.  A  Part  to  Play 

Even  the  best-laid  plans  leave  a  troubling  doubt  in  the  minds  of  some 
people.  Until  people  are  given  a  part  to  play  they  will  feel  left  out  and  find  it  difficult  to 
make  a  new  beginning.  Bridges  indicates  that  people  should  have  two  parts  to  play.  First 
they  need  to  see  their  role  and  their  relationship  to  others  in  the  new  scheme  of  things. 
They  should  also  have  a  role  in  dealing  effectively  with  the  transition  process  itself.  The 
easiest  way  to  do  this  is  to  be  sure  that  everyone  has  a  place  on  a  planning  task  force, 
climate  survey  group,  problem-solving  circle  or  transition  monitoring  team.  If  this  is  not ' 
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possible,  set  up  formal  input  systems  for  such  groupings  so  that  each  person  has  at  least 
an  indirect  part  to  play  in  the  transition  management  process.  Commitment  to  the  new 
beginning  is  enhanced  by  involving  as  many  people  as  possible  in  the  plan.  Everyone 
who  plays  a  part  is  implicitly  implicated  in  the  outcome.  (Bridges,  1991) 

4.  Reinforcing  the  New  Beginning 

Bridges  indicates  that  beginnings  that  are  not  reinforced  will  revert  to  chaos  when 
confronted  by  the  continuing  stream  of  changes  that  are  sure  to  come  along.  To  reinforce 
the  new  beginning,  senior  management  must  ensure  that  they  are  sending  consistent 
messages  about  the  importance  and  the  priority  of  the  change.  If  you  tell  people  they  need 
to  do  five  new  things  but  don't  remove  anything  from  their  list  of  tasks,  you  are  sending 
conflicting  messages  (Bridges  p.61).  Conflicting  messages  are  confusing  in  their  own 
right  and  dso  provide  people  with  excuses  to  argue  that  the  new  beginning  isn't  for  real. 
(Bridges,  1991) 

It  is  common  and  disastrous  to  tell  people  to  act  and  react  in  new  ways  and  then  to 
reward  them  for  the  old  actions  and  reactions.  People  have  to  feel  that  they  are  better  off 
for  having  changed  their  attitudes  and  behavior,  and  if  they  don't  you'd  better  look  at  your 
reward  system.  (Bridges,  1991) 

E.  SUMMARY 

A  number  of  significant  issues  related  to  change  implementation  were  presented 
in  the  work  of  Bridges  (1991),  Dalziel  and  Schoonover  (1988),  and  Pasmore  (1988). 
Although  their  approaches  differ,  there  is  some  overlap  in  their  coverage  of  certain 
concepts.  The  overarching  themes  of  the  literature  addressed  assessing  an  organization's 
readiness  for  change,  change  implementation  strategies,  and  sustaining  change. 
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1.  Organizational  Preparation 

The  literature  indicates  the  importance  of  assessing  the  organization's  preparation 
for  the  planned  change.  Internal  dimensions  of  organizational  preparation  include  the 
culture  and  attitudes  of  the  workforce,  the  workforce's  recognition  of  the  problem  that  the 
change  addresses,  and  management  support.  These  internal  dimensions  of  organizational 
preparation  can  be  managed  through  communication,  selling  the  problem,  and  the  use  of 
information-gathering  techniques  to  assess  workforce  concerns.  External  dimensions  of 
organizational  preparation  reflect  the  adequacy  of  open  systems  planning  or  a  search  of 
environmental  factors  such  as  customers,  competitors,  market  conditions,  and  legislation. 
Managers  must  have  a  clear  understanding  of  the  impact  of  environmental  factors  on  the 
implementation  of  the  planned  change.  Scanning  the  environment  is  a  technique  for 
assessing  and  working  within  the  constraints  of  the  organization’s  external  environment. 

2.  Process  Improvement  Strategy 

The  literature  review  highlighted  the  significant  attributes  of  a  change  plan.  The 
plan  should  be  detailed  and  targeted  at  all  levels  of  the  organization  and  encourage 
involvement  by  as  many  people  in  the  organization  as  possible.  The  plan  should  ensure 
quick  successes  initially  and  align  with  the  business  goals  of  the  organization.  In 
addition,  changes  should  be  piloted  on  a  representative  project  where  possible. 

3.  Sustaining  Change 

The  literature  emphasizes  the  importance  of  establishing  a  culture  where  "change 
is  the  norm"  in  an  organization  for  sustaining  change.  Techniques  for  establishing  such  a 
culture  include  encouraging  innovation,  not  punishing  failure,  ongoing  training  and 
development,  and  reward  systems  that  are  consistent  with  a  learning  culture. 
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In  the  section  that  follows  data  from  software  process  improvement  case  studies, 
technical  reports,  and  other  papers  will  be  examined  within  the  context  of  the 
aforementioned  issues. 

F.  SOFTWARE  PROCESS  IMPROVEMENT  CASES 

1.  Introduction 

In  this  section  the  software  process  improvement  efforts  of  four  organizations  are 
reviewed: 

•  Boeing  Defense  and  Space  Group  (D&SG),  Space  Transportation  Systems 
(STS) 

•  Hughes  Aircraft  Software  Engineering  Division  (SED) 

•  The  Software  Engineering  Laboratory  at  NASA  Goddard  Space  Right  Center 

•  Raytheon  Electronic  Systems  Organization 

These  organizations  were  selected  for  review  because  of  their  outstanding 
reputation  for  software  process  improvement  and  because  of  the  rich  data  available  on 
their  efforts.  Hughes  (1997),  NASA/GSFC  (1994),  and  Raytheon  (1995)  are  winners  of 
the  TF.F.F.  Software  Process  Achievement  Award,  and  Boeing  STS  is  one  of  only  four 
CMM  level  5  organizations  listed  on  the  current  organization  maturity  profile  maintained 
by  SEI.  It  is  not  the  intent  of  this  thesis  to  enumerate  all  process  improvement  activities 
of  these  organizations;  rather  the  intent  is  to  derive  those  best  practices  that  are  likely  to 
contribute  to  the  greater  knowledge  of  effective  software  process  improvement  activities. 

2.  Boeing  Space  Transportation  Systems 

In  July,  1996  the  Boeing  Defense  and  Space  Group  (D&SG),  Space 
Transportation  Systems  (STS)  organization  achieved  a  level  5  rating  on  the  SEI's  CMM. 
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A  series  of  articles  relating  Boeing’s  experiences  in  achieving  CMM  level  5  provide 
insight  into  the  process. 

George  Yamamura,  Software  Engineering  Process  Manager  for  Boeing  STS, 
attributes  Boeing’s  success  in  process  improvement  to  an  approach  that  is  responsive  to 
business  goals  and  an  emphasis  on  the  human  issues  of  process  improvement 
(Yamamura,  1998).  In  a  presentation  given  at  the  March,  1998  SEPG  conference, 
Yamamura  recommended  conducting  surveys  to  assess  worker’s  perspectives  on  issues  of 
importance,  levels  of  satisfaction,  and  areas  of  improvement.  He  suggests  using  employee 
inputs  in  conjunction  with  an  analysis  of  the  situation  to  develop  a  change  strategy. 

Boeing’s  Management  Goal  Framework  is  a  practical  example  of  clarifying  plans 
as  prescribed  by  Dalziel  and  Schoonover  (1988).  It  provides  a  roadmap  for  the  business 
case.  It  starts  with  the  organization’s  business  goals  and  breaks  them  down  in  successive 
levels  until  they  are  translated  to  process  activities  at  the  project  level. 

Boeing's  four  step  formula  for  success,  RSST:  Right  thing.  Small  steps.  Simple, 
and  Timing  can  be  summarized  as  understanding  the  problem  within  the  context  of  the 
environment,  looking  for  simple,  appropriate  solutions,  and  implementing  in  small  steps 
to  gain  quick  successes. 

Boeing’s  mechanism  for  evaluating  candidate  process  changes  (shown  in  figure 
2.2)  is  of  particular  significance  to  this  thesis.  It  provides  a  framework  for  satisfying 
some  of  the  key  requirements  for  successful  change  implementation,  including  gaining 
management  support,  piloting  the  change,  monitoring,  and  feedback.  (Kness  &  Satake, 
1997) 


38 


Figure  2.3  Boeings  Process  Improvement  Process 
(Adapted  from  Kness  &  Satake,  1997) 

After  an  improvement  plan  is  made  and  initial  management  support  is  gained,  the 
potential  process  improvement  initiative  is  piloted  on  a  representative  software 
development  project.  Once  the  benefits  of  the  piloted  improvement  are  evaluated,  a  gate 
review  is  conducted  with  management  to  determine  if  more  widespread  implementation 
is  warranted.  After  which,  the  improvement  may  be  implemented  on  other  projects.  The 
implementation  of  the  improvement  is  then  closely  monitored.  A  working  group  is 
established  to  address  issues  of  refinement  of  the  improvement  and  training  and  to 
develop  lessons  learned.  Boeing  indicates  that  this  approach  helps  prevent  waste  of 
resources  in  process  changes  that  do  not  achieve  the  desired  results.  (Kness  &  Satake, 
1997) 

Boeing’s  approach  to  SPI  is  consistent  with  thdoretical  concepts  of  preparing  the 
organization  for  change  (via  information  gathering  and  an  awareness  of  the  environment) 
and  sound  change  impleirientation  strategies  (via  alignment  with  business  goals  and 
clearly  defined  plans). 
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3.  Hughes  Aircraft  Software  Engineering  Division 

Hughes  Software  Engineering  Division  (SED),  formed  in  1978,  primarily  works 
on  US  Defense  Department  contracts.  It  employs  about  500  professionals.  Of  these,  41 
percent  have  10  to  20  years  experience  in  software  and  12  percent  have  20  or  more  years 
experience.  In  1990,  SED  was  assessed  at  CMM  level  3  by  the  Software  Engineering 
Institute.  The  two-year  program  of  improvements  cost  $400,000.  Hughes  found  that  the 
investment  improved  working  conditions,  employee  morale,  and  the  performance  of  the 
SED  as  measured  in  project  schedule  and  cost.  Hughes  estimates  the  resulting  annual 
savings  to  be  about  $2  million.  (Humprey,  Snyder,  &  Willis,  1991)  The  factors  that 
contributed  to  Hughes  success  include  top  management  support,  alignment  with  business 
goals,  employee  involvement,  and  a  comprehensive  technology  transfer  function. 

Hughes  realized  that  implementation  of  the  SPI  action  plan  as  originally  drawn  up 
would  require  additional  funding.  Consequently  support  and  involvement  of  top 
management  (who  reviewed  the  action  plan  prior  to  approving  additional  funding)  was  an 
early  requirement. 

The  action  plan  listed  the  goals  of  the  SPI  effort  and  put  the  action  plan  in  the 
context  of  a  Ground  Systems  Group  Organizational  Improvement  Strategy.  The  action 
plan  also  included  an  estimation  of  the  labor  required  for  implementation.  Additionally 
quality  indicators  (i.e.,  error  and  detect  counts)  were  identified,  collected  and  presented  to 
senior  management  in  monthly  briefs  on  the  health  of  each  project.  (Humphrey  et.  al., 
1991) 

Hughes  credits  the  creation  of  a  Head  of  Technology  Transfer  function  as  the 
most  profound  action  in  the  entire  improvement  process. 
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Among  other  things,  the  head  of  technology  transfer  coordinated  self- 
assessments,  developed  a  questionnaire  glossary,  became  the  local  expert  in  the 
SEI  maturity  questionnaire,  became  a  member  of  the  Software  Productivity 
Consortium’s  technology-transfer  advisory  group,  developed  an  SPC  technology 
transfer  plan,  briefed  senior  management  on  the  state  of  process  maturity, 
maintained  a  database  of  technology  used  on  each  project  and  an  awareness  of 
what  technology  each  project  needed,  facilitated  technology  transfer  among 
projects,  and  a  special-interest  group  on  process  improvement,  supported  the 
corporate  wide  technology  transfer  program,  served  on  the  practices  and 
procedures  change  review  board,  the  training  policy  committee,  and  the 
technology  steering  committee  (Humprey  et  al.,  p.l8). 


While  it  is  perhaps  more  impressive  that  one  person  could  effectively  handle  so 
many  tasks,  the  detailed  breakdown  of  these  tasks  reflects  the  meticulous- nature  of 
Hughes  SPI  plan. 

Hughes’  SPI  implementation  strategy  is  illustrative  of  a  number  of  the  theoretical 
concepts  for  successfully  implementing  change.  Their  early  involvement  of  top 
management  and  emphasis  on  employee  feedback  are  practices  consistent  with  Dalziel 
and  Schoonover’s  (1988)  principles  for  preparing  an  organization  for  change.  The  many 
tasks  of  the  Head  of  Technology  Transfer  seem  to  be  aimed  at  optimizing  the  fit  between 
new  technologies  and  the  people  as  Pasmore  (1988)  advocates.  These  tasks  also  provide 
mechanisms  for  gauging  and  guiding  the  impact  of  technical  changes  on  the  organization, 
consistent  with  Bridges  (1991)  guidance  on  managing  the  transition.  This  SPI  strategy 
allowed  Hughes  to  progress  from  CMM  level  2  in  1987  to  a  strong  level  3  in  1990.  In 
1997  Hughes  was  the  recipient  of  the  Software  Process  Achievement  Award. 

4.  The  Software  Engineering  Laboratory  at  NASA  Goddard  Space  Flight 
Center  (GSFC) 

A  partnership  between  NASA/GSFC,  the  University  of  Maryland,  and  Computer 
Sciences  Corporation  forms  the  Software  Engineering  Laboratory  which  was  created  in 
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1976  for  the  purpose  of  understanding  and  improving  the  overall  software  process  and 
products  that  were  being  created  within  Flight  Dynamics  Division  (FDD)  of  NASA  SEL. 
SELs  continual  experimentation  with  software  process  has  yielded  an  extensive  set  of 
empirical  studies  that  has  guided  the  evolution  of  standards,  policies,  management 
practices,  technologies  and  training  within  the  organization.  Trends  have  been  observed 
from  as  far  back  as  17  years.  (McGarry,  Pajerski,  Page,  &  Waligora,  1994)  NASA  SEL  is 
the  first  organization  to  win  the  IEEE  Computer  Society  Software  Process  Achievement 
Award.  This  organization  could  represent  the  pinnacle  of  SPI  implementation.  Factors 
that  contribute  to  their  success  include:  a  bottom-up  approach  to  SPI  that  emphasizes 
local  goals,  characteristics,  and  product  attributes,  alignment  with  business  objectives, 
and  a  learning  culture. 

Incorporating  the  key  concept  of  change  guided  by  development  project 
experiences  (McGarry  et.  al.,  1994),  the  SEL  defined  a  standard  paradigm  to  illustrate  its 
concept  of  software  process/product  improvement.  This  paradigm  is  a  three-phase  model 
(figure  2.4)  which  includes  the  following  steps: 

1.  Understanding:  Improve  insight  into  the  software  process  and  its  products 
by  characterizing  the  production  environment,  including  types  of  software 
developed,  problems  defined,  process  characteristics,  and  product 
characteristics. 

2.  Assessing:  Measure  the  impact  of  available  technologies  and  process 
change  on  the  products  generated.  Determine  which  technologies  are 
beneficial  and  appropriate  to  the  particular  environment  and,  more 
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importantly,  how  the  technologies  (or  processes)  must  be  refined  to  best 
match  the  process  with  the  environment. 

3.  Packaging:  After  identifying  process  improvements,  package  the 
technology  for  application  in  the  production  organization.  This  includes 
the  development  and  enhancement  of  standards,  training,  and  development 
policies.  (McGarryet.  al.,  1994) 


Figure  2.4  NASA  SEL  3  Phase  Model  (Adapted  from  McGarry  et.  al.,  1994) 


Change  is  driven  by  product  and  not  merely  process  alone  (McGarry  et.  al., 
p.40).  SEL's  philosophy  that  if  a  changed  process  has  no  positive  impact  on  the  product 
generated  then  there  is  no  justification  for  making  the  change,  closely  links  SPI  efforts 
with  local  business  objectives.  The  SEL  process  improvement  strategy  emphasizes  a 
baseline  understanding  of  the  local  software  process,  products,  and  goals.  SEL 
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distinguishes  their  "bottom-up"  approach  from  the  typical  top-down  approach  in  the 

following  way.  (McGarry  et.  al.,  1994) 

The  top-down  approach  is  based  on  the  assumption  that  there  are  generalized, 
universal  practices  that  are  required  and  effective  for  all  software  development, 
and  that  without  these  practices,  an  organization's  process  is  deficient.  This 
concept  does  not  take  into  account  the  performance  issues,  problems,  and  unique 
software  characteristics  of  the  local  organization.  The  goals  and  characteristics  of 
the  local  organization  are  not  the  driving  elements  of  change. 

In  contrast  the  underlying  principle  of  the  SEL  approach  is  that  "not  all 
software  is  the  same."  Its  basic  assumption  is  that  each  development  organization 
is  unique  in  some  (or  many)  aspects.  Because  of  that,  each  organization  must  first 
completely  understand  its  local  software  business  and  must  identify  its  goals 
before  selecting  changes  meant  to  improve  software  processes.  If,  based  on  that 
understanding,  change  seems  called  for,  then  each  change  introduced  is  guided  by 
"experience" — not  by  a  generalized  set  of  practices.  (McGarry  et.  al  1994) 

Process  change  has  been  infused  as  a  standard  business  practice  at  NASA  SEL. 

Each  production  project  in  the  Flight  Dynamics  Division  is  considered  an  opportunity  for 

the  SEL  to  expand  its  knowledge  of  process  understanding  and  improvement.  SEL 

analysts  identify  software  process  modifications  that  they  hypothesize  are  likely  to 

improve  the  resultant  product  and  then  design  an  experiment  to  test  the  hypothesis.  The 

SEL  conducts  three  general  types  of  analysis,  all  of  which  are  active  continually  in  the 

.  environment.  They  include: 

•  Pilot  studies  of  specific  techniques  and  technologies  on  a  project  or  set  of 
projects. 

•  Studies  of  completed  projects  for  development  and  refinement  of  local  process 
and  product  models. 

•  Trend  analysis  of  completed  projects  to  track  the  impact  of  specific  process 
changes  on  the  environment  as  a  whole.  (McGarry  et.  al.  1994) 
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SELs  emphasis  on  product  improvement  over  process  improvement  ensures  that 
changes  are  compatible  with  existing  organizational  practices  and  plans  as  prescribed  by 
Dalziel  and  Schoonover  (1988).  Their  view  of  every  project  as  a  learning  opportunity  is 
consistent  with  Pasmore's  (1988)  assertion  that  the  greater  the  level  of  experimentation  in 
organizational  plans  the  greater  the  likelihood  that  learning  will  occur  and  lead  to  more 
effective  adaptations  to  the  environment. 

5.  Raytheon  Electronic  Systems 

Raytheon  Company  is  an  international  company  that  operates  in  four  business 
areas:  commercial  and  defense  electronics,  engineering  and  construction,  business 
aviation,  and  major  appliances.  Raytheon  Electronic  Systems  focuses  on  commercial  and 
defense  electronics.  RES  is  a  recently  consolidated  organizational  entity  that  emerged 
from  the  restructuring  of  Raytheon’s  defense  business  in  1995.  (Haley,  Ireland, 
Wojtaszek,  Nash,  &  Dion,  1995) 

In  the  fall  of  1987  prompted  by  the  lack  of  success  in  delivering  software  projects 
on  schedule  and  within  budget,  the  Equipment  Division  of  Raytheon  (now  part  of  RES) 
initiated  a  SPI  effort  known  as  the  Software  Engineering  Initiative.  In  1995  the 
Equipment  Division  was  the  recipient  of  the  Software  Process  Achievement  Award. 
Factors  that  contributed  to  Raytheon’s  success  include:  an  emphasis  on  quantitative 
measures  of  process  improvement,  a  well  defined  implementation  plan,  and  continuous 
feedback. 

A  continuous  concern  that  the  $1  million  annual  expenditure  of  discretionary 
funds  on  SPI  was  not  really  achieving  a  return  sufficient  to  justify  not  spending  the 
money  on  something  else  fueled  Raytheon’s  drive  for  quantitative  measurements  of 
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process  improvements.  Raytheon  measured  the  return  on  investment  on  SPI  based  on:  the 
amount  of  rework,  the  predictability  of  software  development  effort,  and  software 
product  quality.  (Haley,  et.  al.,  1995) 

Raytheon’s  process  improvement  strategy  consists  of  the  organization's  standard 
software  engineering  process  which  is  defined  by  an  underlying  RES  software  policy. 
This  policy  describes  the  set  of  common  software  engineering  practices:  the  “whats”  of 
developing  software,  detailed  procedures  describing  the  how  of  critical  aspects  of 
software  development,  along  with  the  tools  and  the  training  needed  to  make  the 
developers  productive  (Haley  et.  al.  p.3). 

Raytheon’s  SPI  strategy  (Figure  2.5)  includes  feedback  mechanisms,  is 
responsive  to  the  environment  and  provides  guidance  at  all  levels  of  the  organization.  The 
effectiveness  of  existing  procedures  are  measured  and  analyzed,  with  the  results  of  the 
analysis  continuously  fed  back  into  the  process  improvement  effort  (Haley  et.  al  p.l9). 
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The  Standard  Software  Engineering  Process 
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Figure  2.5  Raytheon's  SPI  Strategy  (Adapted  from  Haley  et.  al.,  1995) 

Raytheon's  "Blue  Books"  are  a  three-tiered  set  of  docuinents  which  define 
Raytheon’s  processes  at  the  highest  level  (Software  Engineering  Policy),  the  intermediate 
level  (Software  Engineering  Standards),  and  the  lowest  level  (Detailed  Procedures  and 
Guidelines).  Defining  the  plan  in  this  way  has  enhanced  “buy-in”  to  Raytheon’s 
improvement  process  at  all  levels  of  the  organization.  (Haley  et.  al.,  1995) 

RES  has  extended  its  practice  of  Key  Program  Reviews  of  the  major  projects  in  each 
business  area  (chaired  by  the  General  Manager)  to  include  the  review  of  process 
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improvement  accomplishments  by  each  of  the  functional  areas  within  RES.  This  elevates 
process  improvement  activities  to  the  level  of  project  activities  and  keeps  levels  linked. 

Any  particular  project  uses  the  organization’s  process,  consciously  tailored  to  its 
particular  needs  and  constraints  along  with  its  own  project  software  development  plan, 
the  key  document  binding  the  project  to  the  process.  The  plan  is  typically  constrained  by 
the  contract,  the  statement  of  work,  and  the  requirements  of  the  system  as  specified  by 
the  customer  or  developed  early  on  during  the  system  definition.  As  the  project  software 
engineering  occurs  and  the  specific  process  is  applied,  two  types  of  feedback  take  place. 
At  the  project  level,  the  software  development  plan  is  refined  to  reflect  lessons  learned  in 
early  phases  of  the  development,  and  at  the  organizational  level,  these  lessons  learned 
will  have  an  impact  on  the  process  improvement  activities  and  eventually  lead  to  the 
creation  of  generic  solutions  to  be  added  to  the  organization's  standards.  In  the  meantime, 
the  process  improvement  activities  being  conducted  by  the  initiative  are  benefiting  from 
the  real  time  application  of  these  solutions  on  projects.  The  project  feedback  along  with 
outside  influences  such  as  technology  drivers,  the  marketplace,  corporate  business 
decisions,  customer  initiatives,  all  have  an  impact  on  the  direction  in  which  process 
improvement  will  occur.  (Haley  et.  al.,  1995) 

Raytheon’s  unique  approach  to  training  also  enhances  organizational  “buy-in”. 

All  courses  are  given  during  working  hours,  which  promotes  the  feeling  of  company 
support  and  enhances  morale.  A  detailed  feedback  questionnaire  is  completed  by  the 
student  at  the  completion  of  the  course.  Further,  during  the  transition  phase,  process 
improvement  discussions  examine  the  effectiveness  of  the  training  provided.  (Haley  et. 
al.,  1995) 
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Raj^heon’s  software  development  process  relies  on  pathfinding,  which  is  the 
process  of  identifying  and  testing  tools,  methods,  and  software  components  prior  to  their 
required  use  on  a  project  in  both  the  development  and  test  environment^  Pathfmding  is  a 
key  component  of  risk  management  because  problems  can  be  identified  early,  and 
mitigation  steps  can  be  taken  before  there  is  schedule  or  cost  impact  to  the  project  (Haley 
et.  al.,  p.  15).  Tools  that  are  new  to  Raytheon  are  given  extra  attention  so  as  to  develop 
explicit  procedures  governing  their  use  and  identify  peculiar  behavior  characteristics. 

Raytheon's  three-tiered  set  of  documents  are  consistent  with  Dalziel  and 
Schoonover's  (1988)  recommendation  of  clarifying  plans  as  a  means  of  ensuring 
successful  change  implementation.  Their  awareness  of  environmental  factors  and  the 
influence  that  these  factors  may  have  on  process  improvement  activities  is  in  line  with 
Pasmore  guidance  on  optimizing  social  and  technical  systems  within  the  constraints  of 
the  environment.  Quantitative  measures  of  process  improvements  and  key  process 
reviews  maintain  compatibility  with  organizational  practices  and  plans  as  recommended 
by  Dalziel  and  Schoonover  (1988). 

Over  the  lifetime  of  the  initiative,  rework  involved  in  building  software  has 
undergone  a  reduction  from  about  40%  of  the  development  cost  to  about  10%.  During 
this  same  period,  productivity  of  the  development  staff  has  increased  by  a  factor  of 
almost  2.8,  and  predictability  of  their  development  budget  and  schedule  have  been 
reduced  to  the  range  of  +1-3%.  RES’  ability  and  willingness  to  analyze  the  impact  of  the 
initiative  in  these  business-oriented  terms  has  greatly  influenced  their  success  in 
maintaining  the  ongoing  sponsorship  of  senior  management.  (Haley,  et.  al.,  1995) 
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G.  BEST  PRACTICES 

Based  on  a  review  of  related  change  management  theory  and  software  process 
improvement  technical  reports  and  papers  the  following  best  practices  have  been  derived. 
It  seems  logical  to  group  the  concepts  as  follows:  concepts  related  to  preparing  the 
organization  for  process  improvement,  concepts  related  to  implementing  process 
improvements,  and  concepts  related  to  sustaining  the  improvement  effort. 

1.  Concepts  related  to  preparing  the  organization  for  process  improvement. 

•  Detailed  assessment  of  the  organization’s  preparation  for  change  is  a  logical 
first  step  in  a  software  process  improvement  effort.  Dalziel  and  Schoonover's 
(1988)  five  attributes  of  organizational  readiness  provide  a  framework  for 
assessing  the  challenges  of  the  process  improvement  effort:  1)  history  of 
change,  2)clarity  of  expectations,  3)origin  of  idea  or  problem,  4)  support  of 
top  management,  and  5)  compatibility  of  organizational  goals. 

•  Scanning  the  environment  is  a  critical  step  in  understanding  the  overall 
context  of  the  process  improvement  initiative.  Buy-in  of  key  customers, 
subcontractors,  and  suppliers  is  a  must  for  successful  change  implementation. 
The  search  conference  method  detailed  in  Pasmore's  (1988)  sociotechnical 
systems  perspective  is  one  example  of  a  framework  for  assessing  the 
organization-environment  fit. 

2.  Concepts  related  to  implementing  process  improvements. 

•  Improvements  should  be  integrated  gradually  within  the  context  of  an  overall 
change  design.  A  change  design  that  relates  and  makes  sense  of  changes 
within  the  organizational  context  is  a  means  of  combating  "change  overload". 
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•  Transition  Management  Teams,  consisting  of  personnel  representing  all  levels 
of  the  organization,  provide  a  feedback  mechanism  for  assessing  the  "real 
impact"  of  improvement  activities  and  demonstrate  organizational  concern  for 
the  people  involved  in  the  change.  They  also  enhance  participation,  provide 
linkage  across  levels  and  assign  responsibility  for  change  management. 

•  SPI  initiatives  as  projects.  Treating  SPI  initiatives  like  projects  enhances  the 
alignment  with  the  organizational  goals  and  increases  visibility  of  the  efforts. 

•  Maximize  involvement  in  the  SPI  process.  By  involving  as  many 
organizational  members  as  possible  in  process  improvement  related  groups 
(including  discussion  groups,  transition  management  teams,  training  groups, 
etc.)  we  increase  communication  between  change  agents,  senior  managers, 
and  practitioners.  We  also  enhance  organizational  awareness  and  buy-in  of 
improvement  efforts. 

3.  Concepts  related  to  sustaining  the  improvement  effort. 

•  Piloting  techniques  evaluate  new  technologies  and  methods  prior  to  their 
implementation  organization  wide.  The  concept  of  an  on-going  process  of 
experimentation  to,  improve  the  organization  is  an  important  mechanism  for 
establishing  a  learning  culture. 

•  By  integrating  small  process  improvements  within  the  project  cycle  (guiding 
change  by  development  project  experience)  we  closely  link  the  process 
improvement  efforts  to  the  organization's  business  goals.  Additionally  we 
"piggy-back"  on  the  existing  reward  structures  and  motivations  geared  toward 
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successful  project  releases.  The  process  should  be  reinforced  with  structured 
project  reviews  and  lessons  learned  to  enhance  the  next  release. 

•  Training  in  change  concepts.  Transforming  the  organization  into  a  learning 
organization  is  a  long-term  project.  It  requires  a  commitment  to  life  long 
learning  on  the  part  of  the  individual  and  the  organization.  It  is  obvious  that 
extensive  technical  training  will  be  required  at  all  levels.  However,  training  in 
concepts  of  change  management  and  learning  organizations  is  also  required  to 
institutionalize  the  movement  to  change  as  the  norm. 
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III.  METHODS 


The  research  method  employed  for  this  thesis  is  a  case  study  of  software  process 
improvement  efforts  at  the  Financial  Systems  Activity-Kansas  City  (FSA-KC).  It  relies 
on  multiple  sources  of  data,  including  site  visits,  archival  material,  personal  interviews 
and  a  survey.  This  chapter  begins  with  a  description  of  the  research  site.  Limited 
descriptions  of  Defense  Finance  and  Accounting  Service  (DFAS)  and  DFAS-KC  Center 
are  provided  because  activities  at  FSA-KC  are  profoundly  affected  by  these 
organizations.  The  remainder  of  the  chapter  details  the  data  collection  procedures  cited 
above. 

A.  DESCRIPTION  OF  RESEARCH  SITE 
1.  DFAS 

The  Defense  Finance  and  Accounting  Service  (DFAS)  was  activated  on  January 
15,  1991,  to  reduce  the  cost  and  improve  the  overall  quality  of  Department  of  Defense 
(DoD)  financial  management  through  consolidation,  standardization  and  integration  of 
finance  and  accounting  procedures,  operations,  and  systems.  In  December  1992,  DFAS 
took  over  responsibility  for  all  finance  and  accounting  operations,  and  the  associated  332 
installation  finance  and  accounting  offices  nationwide  and  began  the  consolidation 
process.  Currently,  DFAS  (Figure  3.1)  consists  of  a  headquarters,  five  centers,  and  17 
operating  locations.  DFAS  is  also  actively  consolidating  and  standardizing  the  hundreds 
of  finance  and  accounting  systems  within  DoD.  By  fiscal  2002,  DFAS  will  reduce  the 
number  of  DoD  finance  systems  from  67  in  1996  to  9.  (DFAS,  1997) 
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2.  DFAS  Kansas  City  Center 

The  successor  to  the  Marine  Corps  Finance  Center,  DFAS-Kansas  City  Center 
(DFAS-KC)  continues  to  coordinate  and  supervise  disbursement  of  funds  in  payment  for 
all  active  duty,  reserve  survivor  annuitants,  and  retired  Marines.  In  addition  DFAS-KC 
provides  financing  and  accounting  services  to  DoD  components  worldwide.  DFAS-KC 
depends  on  FSA-KC  for  the  development  of  software  to  provide  these  finance  and 
Accounting  services.  The  total  workforce  at  DFAS-KC  including  civilians,  service 
members,  and  Defense  Accounting  Office  employees  is  close  to  1000.  (DFAS,  1997) 


(Adapted  from  DFAS,  1997) 

3.  DFAS  Realignment 

Four  of  the  six  FSAs  within  DFAS  are  co-located  with  the  centers  that  they 
provide  software  development  services  for.  However,  the  DFAS  command  structure  (pre- 
April  98)  has  all  FSAs  reporting  directly  to  the  Director  Financial  Systems  Organization 
(FSO).  The  FSO  provides  DFAS-wide  infrastructure  services  including  development 
services,  operations  services,  and  business  management  services.  Prior  to  April  1998 
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there  was  a  "customer-service  provider"  relationship  between  the  DFAS  Centers  and  the 
FS  As.  The  Center  Director  was  not  responsible  for  the  performance/viability  of  the  FSA. 
See  Figure  3.2. 


Figure  3.2  DFAS  Prior  to  Realignment  (Adapted  from  DFAS,  1998) 


Effective  5  April  1998  control  of  FSAs  was  transferred  from  the  FSO  to  the 
DFAS  center  directors.  The  move  was  initiated  to  mitigate  "customer-service  provider" 
problems  by  placing  responsibility  for  both  functions  (information  systems  and  financial 
operations)  under  one  manager.  Another  reason  for  the  realignment  is  to  leverage  the 
information  systems  expertise  (which  is  concentrated  in  the  FSA/FSO  structure)  across 
the  DFAS  Center.  The  FSA  Director  has  become  the  DFAS  Center  Director  for 
Information  and  Technology  and  the  FSO  has  been  renamed  the  Infrastructure  Services 
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Organization  (see  Figure  3.3).  DFAS  is  taking  a  phased  approach  to  the  realignment.  The 
new  command,  control,  and  staff  relationships,  and  revisions  to  policy  and  procedures 
will  evolve  over  the  next  few  months  with  October  1,  1998  as  the  target  date  for 
completion  of  realignment. 


Figure  3.3  DFAS  After  Realignment  (Adapted  from  DFAS,  1998) 


4.  FSA-KC 

FSA-KC  provides  resources  and  support  in  performing  a  variety  of  software 
development  services  for  the  US  Marine  Corps,  and  the  DFAS  Center.  FSA’s  primary 
software  development  arenas  are  service  unique  pay  and  personnel  systems  (for  the 
Marine  Corps)  and  “purple  systems”  or  systems  that  are  developed  for  DoD-wide  use. 
(FSA,  1997)  The  mainstay  of  the  work  at  FSA  involves  the  scheduled  implementation  of 
improvements  and  revisions  to  software.  These  “software  releases”  normally  occur  in 
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April  and  October.  The  organization  develops  and  maintains  a  growing  number  of 
automated  information  systems  (AIS).  At  this  time  FSA’s  major  automated  information 
systems  are  the  Marine  Corps  Total  Force  System  (MCTFS)  and  DoD's  Standard 
Accounting  and  Budgeting  Reporting  System  (SABRS).  They  account  for  40%  and  19%, 
respectively  of  the  organization's  software  development  effort.  The  current  organization 
chart  for  FSA  appears  in  Figure  3.4.  (FSA,  1997) 

FSA-KC  is  co-located  with  the  director  and  staff  of  DFAS-KC  Center.  However 
prior  to  April  1998  the  relationship  between  the  DFAS-KC  and  FSA  was  a  customer- 
service  provider  relationship.  Effective  5  April  1998  control  of  FSA-KC  was  transferred 
from  the  Director  FSO  to  the  Director  DFAS-KC. 


Figure  3.4  Financial  Systems  Agency  (Adapted  from  FSA,  1997) 

5.  Fee-for-Service 


FSA  is  a  revolving  fund  activity.  The  activity  bperates  similar  to  a  not-for-profit 
business.  Customers  are  charged  a  fee  sufficient  for  the  activity  to  recover  the  costs  of 
providing  the  service.  In  this  “fee-for-service”  arrangement,  the  objective  (for  FSA)  is  to 
break  even.  Unit  costs  are  adjusted  annually  based  on  an  agreed  upon  work-load  between 
the  activity  and  customers.  Obviously  in  this  era  of  shrinking  defense  budgets,  FSA's 
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customers  (DoD  and  USMC)  still  maintain  the  traditional  view  of  trying  to  get  the  lowest 
price.  Therefore  lowering  unit  costs  is  a  high  priority  at  FSA. 

6.  Workforce 

FSA  has  a  diverse  workforce  consisting  of  civilians,  contractors,  and  military 
personnel.  The  current  workforce  at  FSA  consists  of  42%  civilian  employees,  32% 
contractors,  and  26%  military  personnel.  The  breakdown  of  employees  is: 

•  Civilian:  147 

•  Military:  93 

•  Contractor:  113 

•  Total:  353  (FSA,  1997) 

7.  Customers 

The  primary  customers  of  the  FSA  are  the  US  Marine  Corps  and  DoD. 
Headquarters  Marine  Corps  (HQMC)  submits  requirements  for  improved  or  upgraded 
services  via  the  Manpower  Information  Systems  Support  Activity  (MISSA),  Kansas  City, 
which  acts  as  the  site  representative.  DoD  initiated  modifications  and  improvements  to 
software  are  routed  via  the  Accounting  and  Finance  Directorates  of  the  DFAS-KC 
Center.  (FSA,  1997) 

8.  Organizational  Ooals 

The  goals  of  the  FSA  as  stated  in  the  document  entitled  “FSA  Director’s  Goals 

and  Commitments”  are: 

•  Provide  exemplary  customer  service. 

•  Reduce  business  costs. 
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•  Continue  process  improvement  to  reach  Level  3  of  the  CMM  for  the  entire 
FSA. 

•  Develop  migration  strategy  to  implement  the  Joint  Technical  Architecture 

9.  History  of  Software  Process  Improvement  Efforts 

The  SPI  infrastructure  for  FSA-KC  was  formed  in  1993.  The  Management 
Steering  Committee  and  the  Software  Engineering  Process  Group  (SEPG)  were  formed 
in  July.  Membership  in  the  management  steering  committee  includes  the  director,  deputy 
director,  senior  managers,  and  the  head  of  the  SEPG.  The  SEPG  is  an  independent  team 
with  4  full  time  members.  The  FSO,  which  has  a  similar  SPI  infrastructure,  provides 
leadership  and  guidance,  especially  in  the  areas  of  technical  architecture  and  process 
improvement.  (FSA,  1998) 

The  SPI  efforts  were  initiated  in  the  organization's  Marine  Corps  Total  Force 
System  (MCTFS)  Division.  The  initial  assessment  of  the  software  engineering  practices 
was  performed  on  1 1  June  1993.  The  purpose  of  the  assessment  was  to  gain  an 
understanding  of  the  division's  software  engineering  practices,  identify  key  areas  for 
process  improvement,  and  develop  an  action  plan.  At  that  time  the  division  was  assessed 
as  CMM  Level  1. 

A  Software  Capability  Evaluation  (SCE)  was  performed  for  MCTFS  on 
September  16-20, 1996,  at  which  time  the  division  was  found  to  satisfy  four  (out  of  6)  of 
the  CMM  Level  2  key  process  areas  (KPAs).  Software  Project  Tracking  and  Oversight 
(SPTO)  was  rated  as  unsatisfied.  The  Software  Subcontract  Management  KPA  was  not 
evaluated.  A  second  SCE  was  performed  on  TFS  on  January  27-  January  30, 1997  to 
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evaluate  the  SPTO  key  process  area.  Having  satisfied  all  requirements  for  the  SPTO  key 
process  area,  MCTFS  was  assessed  at  CMM  Level  2. 

10.  Current  SPI  Efforts 

FSA-KC’s  software  process  improvement  efforts  are  ongoing.  Currently  the 
MCTFS  program  is  assessed  at  CMM  level  2  and  the  SABRS  program  is  at  CMM  level 
1.  SPI  efforts  in  the  SABRS  AIS  were  initiated  in  1997  but  were  quickly  overcome  by 
eventS“Specifically  a  backlog  of  requests  for  software  modification  and  development  in 
the  upcoming  releases.  The  FSA-KC  organization  is  in  the  planning  stages  of  a  level  3 
initiative  that  they  expect  to  complete  by  October  1999.  At  that  time  it  is  expected 
MCTFS  will  be  at  CMM  Level  3  and  SABRS  will  be  at  CMM  Level  2. 

B.  DATA  COLLECTION  PROCEDURES 

1.  Process  Improvement  Survey 

A  40-question  survey  was  designed  to  assess  the  challenges  facing  the  FSA’s 
process  improvement  initiative.  Survey  content  was  designed  to  capture  the  following 
key  aspects  of  implementing  change  as  indicated  in  the  literature  review. 

•  Level  of  Buy-in  Across  the  Organization 

•  Management  Support 

•  Clarity  and  Consistency  of  Organizational  Goals 

•  Alignment  with  Organizational  Goals  and  Practices 

•  Encouragement  of  Innovation 

The  survey  was  designed  to  be  administered  to  the  entire  FSA  organization  during 
the  author's  second  site  visit.  Demographic  data  was  requested  to  classify  results  by 
division,  role  (senior  management,  management,  and  section  personnel),  software 


60 


experience,  and  time  employed  at  the  organization.  The  survey  was  administered  via 
organizational  e-mail  with  the  use  of  a  commercial  survey  software  tool.  The  survey 
software  provided  a  means  to  collect  and  statistically  analyze  responses. 

Due  to  a  requirement  for  prior  approval  by  the  Union  and  to  other  surveys 
scheduled  to  be  administered  during  the  same  time  frame,  the  process  improvement 
survey  was  administered  a  month  after  the  second  site  visit.  The  survey  was  sent  out 
without  the  demographic  data.  There  were  72  responses  from  a  possible  250  civilian  and 
military  employees  for  a  29%  response  rate.  Appendix  A  contains  the  complete  survey. 

2.  Archival  Data  Collection 

During  the  site  visits  volumes  of  data  relating  to  the  FS  A  organization  and  its 
software  process  improvement  efforts  were  obtained.  Documentation  on  general 
procedures,  SPI  activities,  and  Benchmark  studies  provided  valuable  data  on  the 
functioning  of  this  complex  organization,  the  history  of  its  SPI  efforts,  and  its  current 
plans  for  implementation  of  SPI  initiatives.  The  FSO’s  Post  Implementation  Review  of 
Level  2  Implementation  (December  1997)  provided  particularly  rich  data  on  employee 
attitudes  about  process  improvement.  The  survey  consists  of  16  questions  (12  rating 
questions  and  4  open-ended  questions).  The  questions  address  issues  relative  to  the 
support  provided  for  level  2  SPI  activities.  The  survey  was  administered  at  all  six  FSAs, 
and  across  all  levels  of  the  organizations.  Data  from  this  report  are  analyzed  in  the 
chapter  that  follows.  Appendix  B  contains  the  complete  survey. 

3.  Personal  Interviews 

Semi-stractured  interviews  were  conducted  during  both  visits  to  FSA-KC.  The 
purpose  of  the  interviews  was  to  assess  organizational  buy-in,  senior  management 


61 


support,  and  general  attitudes  and  perceptions  of  software  process  improvement.  A  total 
of  14  individuals  were  interviewed  including:  FSA  Director,  Project  Officers  for  MCTFS 
and  SABRS,  3  Division  Heads  (out  of  the  7  that  were  available),  6  Branch  Heads  (out  of 
13  that  were  available),  and  on  site  customer  representatives  for  the  US  Marine  Corps 
and  the  DFAS  Center.  Each  interview  was  conducted  individually,  lasting  approximately 
15  minutes  on  average,  and  notes  were  taken.  Interview  results  were  analyzed  for 
common  themes  and  unique  perspectives.  Appendix  C  lists  the  interview  questions. 

4.  Site  Visits 

Two  visits  to  FSA-KC  were  conducted  over  the  course  of  the  thesis  research.  The 
initial  visit  took  place  from  23-27  March  1998  and  the  follow  on  visit  was  from  20-23 
July  1998.  The  purpose  of  the  initial  visit  was  to  become  familiar  with  the  organization, 
including  its  structure,  the  software  process  improvement  infrastructure,  and  general 
attitudes/perceptions  about  software  process  improvement.  During  this  time  the  author 
shadowed  the  SEPG,  attending  number  of  meetings  on  the  status  of  SPI  efforts  within 
FSA,  and  also  observing  the  SEPG’s  brief  on  software  process  improvement  to  the  new 
DFAS  Center  Director.  Additionally,  the  author  reviewed  documents  and  conducted 
interviews. 

On  the  subsequent  visit,  the  goal  was  to  focus  more  specifically  on  gathering  data 
related  to  the  research  question:  What  are  the  challenges  to  the  software  process 
improvement  efforts  at  FSA?  Again,  the  author  spent  a  good  portion  of  the  week 
shadowing  the  SEPG,  reviewing  documents,  conducting  interviews  and  asking  questions. 
These  data,  along  with  the  more  formal  interviews  are  used  in  conjunction  with  survey 
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data  from  FS  A  and  FSO  to  identify  the  perceptions  of  SPI,  its  purpose,  progress  and 
challenges. 
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IV.  RESULTS  AND  ANALYSIS 

A.  INTRODUCTION 

In  this  chapter  the  results  of  a  process  improvement  survey,  archival  material, 
personal  interviews,  and  general  impressions  gained  from  site  visits  are  presented  and 
analyzed.  These  data  are  used  to  extract  themes  relevant  to  challenges  of  the  FSA’s 
software  process  improvement  efforts.  This  chapter  concludes  with  a  summary  of  the 
results  and  their  implications. 

B.  PROCESS  IMPROVEMENT  SURVEY 

A  forty-question  survey  was  designed  to  analyze  the  dimensions  of  the  challenges 
to  the  FSA’s  process  improvement  initiative.  The  survey  was  administered  via 
orgMizatiOnal  e-mail  with  the  use  of  a  commercial  survey  software  tool.  There  were  72 
responses  from  a  possible  250  civilian  and  military  employees,  for  a  29%  response  rate. 
Due  to  the  limited  sample  size  and  the  lack  of  demographic  information,  the  reader  is 
cautioned  that  survey  results  may  not  be  representative  of  the  population.  The  survey  is 
of  value  to  the  extent  that  it  may  be  used  with  other  data  gathering  techniques  in  analysis 
of  the  FSA  organization.  Survey  questions  cover  a  broad  range  of  organizational  structure 
and  culture  issues.  However,  the  most  relevant  results  are  found  in  the  responses  to 
questions  regarding  the  following  issues: 

•  Clarity  and  consistency  of  organizational  goals. 

•  Perceptions  on  the  need  for  process  improvement. 

•  Compatibility  of  process  improvement  with  organizational  practices. 

•  The  extent  to  which  innovation  is  encouraged. 
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Survey  questions  and  statements  addressing  the  aforementioned  issues  and 
summaries  of  responses  by  percentage  are  provided  in  Tables  4.1  through  4.7.  The 


complete  results  are  provided  in  Appendix  A. 


Responses  to  questions  are  rated  on  one  of  two  five  point  scales  as  shown  below. 


Selection  Rating 

Very  Much  5 

Some  what  4 

Very  Little  3 

Not  at  all  2 

Don't  know  1 


Selection 

Strongly  Agree 

Agree 

Disagree 

Strongly  Disagree 
Don’t  know 


A  level  5  response  (Very  much  or  Strongly  Agree)  is  considered  a  high  rating  reflective 
of  a  more  definitive  or  positive  response  to  the  question  or  statement.  A  level  2  response 
(Not  at  all  or  Strongly  disagree)  is  considered  a  low  rating  indicating  a  more  definitive 
negative  response  to  the  question  or  statement. 

1.  Clarity  and  Consistency  of  Organizational  Goals 

The  questions  regarding  organizational  goals  are  designed  to  assess  the  clarity  and 
consistency  of  orgcinizational  goals  throughout  the  organization.  Dalziel  and  Schoonover 
(1988)  discuss  compatibility  of  a  planned  change  with  organizational  goals  as  a  primary 
factor  in  getting  an  organization  ready  for  the  change.  A  good  starting  point  is  ensuring 
that  current  goals  are  understood  and  shared  by  all. 

The  first  series  of  questions  (Table  4.1)  assess  respondents  perceptions  on  the 
extent  to  which  each  factor — customer  satisfaction,  product  quality,  controlling  cost  and 
schedule — is  a  measure  of  the  organization’s  success.  The  second  series  of  questions 
regarding  organizational  goals  (Table  4.2)  ask  respondents  to  indicate  the  extent  to  which 
each  of  the  aforementioned  factors  is  a  measure  of  success  in  their  immediate  work  area. 
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To  what  extent  Is  custonner  satisfaction  used 

as  a  measure  of  success  at  FSA? 

Very  Much 

45 

Some  what 

37 

Very  Little 

4 

Not  at  all 

0 

Don't  know 

14 

To  what  extent  is  product  quality  used 

as  a  measure  of  success  at  FSA? 

39 

37 

7 

0 

17 

To  what  extent  is  controlling  cost  used 

as  a  measure  of  success  at  FSA? 

45 

30 

8 

0 

17 

To  what  extent  Is  schedule  used 

as  a  measure  of  success  at  FSA? 

48 

31 

6 

1 

14 

Table  4.1  Ratings  of  Goal  Clarity  and  Consistency  in  FSA  (%) 


Responses  to  the  first  series  of  questions  are  almost  equally  distributed  with 
respondents  indicating  that  all  of  the  factors  are  measures  of  organizational  success. 
Between  45%  and  48%  of  respondents  gave  level  5  responses  for  cost  and  Schedule  as 
measures  of  the  organizations  success.  Although  there  were  slightly  more  level  5 
responses  for  schedule  as  a  measure  of  organizational  success,  none  of  the  factors  really 
emerges  as  the  primary  measure  of  success. 

In  contrast,  responses  to  the  second  series  of  questions  regarding  measures  of 
success  within  the  immediate  work  area  indicate  that  respondents  view  product  quality 
(69%  selected  “very  much”)  as  a  primary  measure  of  success  within  their  immediate 
work  area.  Both  customer  satisfaction  and  schedule  also  received  level  5  ratings  by  over 
50%  of  the  respondents.  Cost  is  reported  less  frequently  as  a  significant  measure  of 
success  in  the  work  area. 
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To  what  extent  is  customer  satisfaction  used 

as  a  measure  of  success  in  your  immediate  work  area? 

Very  Much 

59 

Some  what 

28 

Very  Little 

6 

Not  at  all 

1 

Don't  know 

6 

To  what  extent  is  product  quality  used 

as  a  measure  of  success  in  your  immediate  work  area? 

69 

23 

4 

1 

3 

To  what  extent  is  controlling  cost  used 

as  a  measure  of  success  in  your  immediate  work  area? 

35 

38 

15 

3 

8 

To  what  extent  Is  schedule  used 

as  a  measure  of  success  in  your  Immediate  work  area? 

52 

28 

11 

1 

7 

Table  4.2  Ratings  of  Goal  Clarity  and  Consistency  in  Work  Area  (%) 


The  results  in  Table  4. 1  and  Table  4.2  indicate  that  respondents  are  more  aware  of 
the  measures  of  success  in  their  immediate  work  area  than  they  are  of  the  measures  of 
success  for  the  organization.  This  is  indicated  by  the  higher  percentage  of  “Don’t  know” 
responses  to  the  first  series  of  questions.  While  this  is  an  expected  outcome,  the 
difference  between  respondents  perceptions  of  the  importance  of  product  quality  as  a 
measure  of  success  within  their  immediate  work  area  and  the  importance  of  product 
quality  to  the  success  of  the  organization  is  significant.  The  results  suggest  that 
employees  believe  that  their  efforts  at  ensuring  high  product  quality  may  be  at  odds  with 
the  organization’s  other  goals.  Customer  satisfaction  and  reducing  cost  are  two  of  the 
stated  goals  of  the  FSA  Director.  (FSA,  1997)  However,  these  factors  don't  emerge  as  the 
primary  measures  of  organizational  success  in  this  survey. 

The  results  in  Table  4.1  and  4.2  are  indicators  of  the  organization’s  goals  to  the 
extent  that  perceived  measures  of  success  are  an  enactment  of  such  goals.  The 
implications  are  that  the  organization’s  goals  are  not  sufficiently  well  understood  and 
shared  by  all.  Additionally,  senior  management  at  FSA  may  be  bypassing  a  great 
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opportunity  to  facilitate  adoption  of  process  improvement  efforts  by  not  building  on  the 
employee  motivation  for  product  quality.  The  discontinuity  between  employee 
perceptions  of  measures  of  success  in  their  work  areas  and  organizational  measures  of 
success  may  suggest  that  employees  do  not  understand  how  their  efforts  contribute  to  the 
organization's  success.  For  example  the  lower  rating  of  importance  of  the  measure  of 
costs  at  the  unit  level  may  reflect  a  lack  of  awareness  of  the  importance  of  this  criteria 
and  an  unclear  understanding  of  unit  level  contribution  to  total  costs.  Moreover,  the 
discontinuity  between  measures  of  success  in  the  work  area  and  organizational  measures 
of  success  may  signal  that  employees  feel  that  the  “product  quality”  indicator  of  success 
that  dominates  at  the  work  unit  level  is  not  valued  as  significantly  at  the  organizational 
level. 

2.  The  Need  for  Process  Improvement 

Achieving  a  CMM  Level  3  rating  is  a  stated  goal  of  the  FS  A  Director.  Statements 
regarding  FSA’s  primary  motivation  for  initiating  process  improvement  are  designed  to 
assess  employee  perceptions  of  the  need  for  process  improvement.  (FSA,  1997)  The 
responses  shown  in  Table  4.3  indicate  some  confusion  regarding  FSA’s  primary 
motivation  for  implementing  process  improvement  efforts.  It  is  surprising  that  only  30% 
feel  strongly  (as  indicated  by  a  “Very  much”  (5)  response)  that  product  quality  is  the 
primary  motivation  for  the  organization’s  process  impfovement  efforts.  Additionally,  the 
results  suggest  that  a  large  percentage  of  respondents  don't  know  what  the  organization's 
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primary  motivation  for  initiating  SPI  efforts  is. 


Very  Much 

Some  what 

Very  Little 

Not  at  all 

Don't  know 

In  the  future  a  CMM  level  3  certification  will  be  required  for  DoD 

software  developers. 

41 

25 

1 

1 

31 

The  requirement  to  initiate  software  process  Improvement  efforts 
came  from  higher  headquarters. 

37 

35 

1* 

1 

25 

Process  improvement  efforts  will  result  in  FSA  producing 
higher  quality  products  in  a  more  timely  fashion. 

30 

32 

17 

3 

18 

Table  4.3  Ratings  of  the  Need  for  Process  Improvement 


Bridges  (1988)  asserts  the  importance  of  a  clear  purpose  for  a  planned  change. 

The  purpose  should  align  with  internal  organizational  characteristics.  The  results  here 
seem  to  reinforce  the  results  of  the  previous  section  that  suggest  the  connection  between 
product  quality  and  process  improvement  efforts  has  not  been  made  clear  to  employees. 

To  the  extent  that  employees  view  SPI  as  mandated  from  higher  authority  without 
acknowledging  the  potential  benefits  to  organizational  performance,  there  will  likely  be 
greater  resistance  to  the  changes  involved  in  SPI  implementation. 

3.  Organizational  Buy-in  and  Support 

Support  of  top  management  is  critical  at  the  initial  stages  and  throughout  the 
process  of  planned  change.  (Dalziel  and  Schoonover,  1988)  Questions  of  how  much 
organizational  buy-in/support  are  designed  to  assess  the  buy-in  and  support  at  the  senior 
management,  middle  management,  and  section  levels  of  the  organization.  Responses  to 
the  questions  have  implications  for  the  level  of  commitment  and  motivation  for  process  ' 
improvement  at  the  various  levels. 
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Very  Much 

Some  what 

Very  Little 

Not  at  all 

Don’t  know 

In  your  judgement  how  much  buy-in  and  support  for 
process  improvement  is  there  among  Senior  Management? 

35 

27 

13 

3 

23 

In  your  judgement  how  much  buy-in  and  support  for 
process  improvement  is  there  among  Middle  Management? 

23 

39 

14 

3 

21 

In  your  judgement  how  much  buy-in  and  support  for 
process  improvement  is  there  among  Section  Personnel? 

17 

42 

18 

6 

17 

Table  4.4  Ratings  of  Organizational  Buy-in  and  Support  (%) 

The  responses  (Table  4.4)  are  very  dispersed  for  each  of  these  questions.  This 
shows  low  consensus.  However,  the  data  do  not  suggest  a  great  deal  of  organizationed 
buy-in/support  at  any  level.  The  data  indicate  that  the  buy-in/support  for  process 
improvement  decreases  as  we  go  from  senior  management  to  middle  management  and  is 
lowest  at  the  section  level.  This  may  indicate  that  the  organization's  efforts  at  achieving 
buy-in  are  aimed  at  managers  and  not  practitioners.  It  may  also  indicate  that  section-level 
practitioners  are  not  sufficiently  involved  in  SPI  activities. 

4.  Compatibility  with  Organizational  Practices 

A  process  is  not  likely  to  be  adopted  when  it  is  not  clear  that  the  change  will 
enhance  current  practices.  Statements  regarding  compatibility  with  organizational 
practices  are  designed  to  determine  the  degree  to  which  process  improvement  efforts  fit 
into  existing  organizational  practices.  See  Table  4.5  for  actual  results. 

Responses  to  the  first  statement  indicate  .that  51%  “don’t  know”  whether  the 
CMM  framework  for  process  improvement  will  address  important  AIS  specific  areas  of 
software  development.  The  remainder  of  responses  are  almost  evenly  split  as  to  whether 
the  CMM  framework  addresses  important  AIS  specific  areas  of  software  development. 

While  the  majority  of  respondents  do  not  agree  that  process  improvement  efforts 
detract  from  day  to  day  operations  (45%),  a  significant  minority  (34%)  does  agree  with 
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the  statement.  Responses  to  the  third  statement,  that  the  organization’s  preoccupation 
with  process  improvement  has  caused  neglect  of  other  issues  are  fairly  evenly  split.  Most 
respondents  (50%)  disagree  with  the  fourth  statement  that  day  to  day  operations  don’t 
leave  time  for  SPI  activities,  but  a  significant  percentage  (35%)  do  agree  with  the 
statement.  Finally,  the  majority  of  respondents  agree  (48%  agree,  and  11%  strongly 
agree)  that  SPI  initiatives  will  make  their  jobs  easier. 

The  responses  to  the  first  question  regarding  whether  the  CMM  addresses  AIS 
specific  areas  are  significant  in  light  of  case  study  findings  on  the  need  to  recognize  and 
adopt  general  CMM  guidelines  to  unique  product/process  or  organizational  requirements. 
The  abundance  of  “don’t  know”  responses  is  another  indicator  of  the  need  for  more 
participation  in  process  improvement  efforts.  Top  down  change  management  will  over¬ 
emphasize  the  generic  rather  than  the  more  useful  locally  specific  process  changes. 

The  responses  to  the  second,  third  and  fourth  statements  indicate  the  challenge 
that  faces  FSA  with  respect  to  balancing  the  process  improvement  initiative  with  day  to 
day  operations.  The  responses  demonstrate  cause  for  concern  that  software  process 
improvement  initiatives  may  be  in  conflict  with  organizational  practices  and  goals.  This 
concern  reinforces  the  question  of  the  importance  of  SPI  goals  relative  to  the  existing 
performance  goals  of  the  organization. 
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• 

Strongly 

Agree 

Agree 

Disagree 

Strongly 

Disagree 

Don’t  know 

Regarding  the  process  improvement  framework,  there  are 
important  AIS  specific  areas  that  the  CMM  does  not  address. 

6 

21 

23 

0 

51 

Process  improvement  efforts  detract  form  day  to  day 

operations. 

17 

17 

42 

3 

21 

Our  preoccupation  with  software  process  improvement  has  caused 
us  to  neglect  other  important  issues  facing  the  organization.  8 

28 

35 

7 

21 

Day  to  day  operations  don't  leave  time  for  software  process 
Improvement  activities. 

4 

31 

44 

6 

15 

Planned  software  process  improvement  initiatives  will  make  my 
job  easier. 

11 

48 

11 

4 

25 

Table  4.5  Ratings  of  Compatibility  with  Organizational  Practices  (%) 

5.  Encouragement  of  Innovation 

Statements  regarding  encouragement  of  innovation  (Table  4.6)  are  designed  to 


assess  the  organization’s  effectiveness  in  creating  an  environment  that  encourages 
employees  to  actively  participate  in  process  improvement.  Bridges  (1991)  believes 
innovation  will  happen  automatically  in  the  neutral  zone  if  people  are  provided  with 
encouragement  and  support.  The  extent  to  which  employees  believe  that  their  ideas  will 
be  acknowledged  and  acted  on  and  that  innovation  will  be  rewarded  is  a  key  indicator  of 
the  climate  of  the  organization  as  it  relates  to  the  enhancement  of  learning  and 


innovation. 


Recommendations/ideas  for  process  improvement  are 

Strongly 

Agree 

Agree 

Disagree 

Strongly 

Disagree 

Don’t  know 

acknowledged  or  acted  on. 

4 

51 

23 

6 

17 

Successful  ideas  or  recommendations  for  process 

improvement  are  rewarded. 

1 

28 

35 

6 

30 

Table  4.6  Ratings  of  Encouragement  of  Innovation  (%) 
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A  majority  of  those  surveyed  agree  that  recommendations  for  process 
improvement  are  acknowledged  and  acted  upon.  However,  there  is  virtually  no  strong 
agreement  with  this  statement.  In  addition,  a  majority  of  respondents  disagree  or  are  not 
aware  that  there  are  any  rewards  for  successful  process  improvement  ideas. 

6.  Challenges  to  Process  Improvement 

Six  statements  are  provided  to  gauge  employee  perceptions  of  the  challenges  to 
FSA’s  process  improvement  efforts.  The  statements  cover  a  range  of  typical  problems 
encountered  in  process  improvement  efforts  as  indicated  in  a  review  of  case  studies  and 
technical  reports.  See  Table  4.7  for  results. 

The  results  indicate  that  respondents  view  all  of  the  statements  as  likely 
challenges  to  the  process  improvement  efforts.  However,  the  responses  indicate  that 
employees  perceive  the  following  as  the  most  likely  challenges  to  the  organization’s 
process  improvement  efforts,  based  on  the  aggregate  of  the  top  2  rating  categories: 

•  Time  and  resource  constraints. 

•  The  possibility  of  being  overcome  by  other  priorities. 

•  Lack  of  customer  support. 

•  Insufficient  developer  training  and  education 
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Very  Much 

Some  what 

Very  Uttle 

Not  at  all 

Don't  know 

Process  improvement  goals  are  not  integrated  into 

currently  established  FSA  goals. 

15 

38 

14 

.8 

24 

Lack  of  customer  support. 

33 

33 

10 

6 

19 

Time  and  resource  constraints. 

36 

41 

9 

1 

13 

Problems  matching  CMM  requirements  to  AIS  specific 

activities. 

9 

30 

33 

3 

26 

Insufficient  developer  training  and  education. 

26 

39 

17 

3 

16 

Overcome  by  other  priorities. 

39 

29 

17 

1 

14 

Table  4.7  Ratings  of  Challenges  to  FSAs  SPI  Efforts  (%) 


These  results  have  implications  to  employee  perceptions  of  the  viability  of  the 
organization’s  SPI  program.  The  above  factors  may  be  an  indicator  that  employees 
perceive  that  process  improvement  is  not  a  high  priority  of  the  organization,  or  that  the 
organization’s  leadership  will  not  ensure  that  the  necessary  resources  and  support  are 
sustained. 

These  responses  provide  a  starting  point  for  senior  management  in  addressing 
employee  attitudes  and  concerns  about  the  current  process  improvement  effort.  The 
possible  perception  that  process  improvement  is  a  passing  fad  will  make  gaining  the 
commitment  and  energy  of  employees  a  challenge. 

C.  ARCHIVAL  MATERIAL 

The  Financial  Systems  Organization's  Level  2  Post  Implementation 
Review  provides  a  wealth  of  data  and  perspectives  on  the  challenges  of  SPI.  As  part  of 
this  review,  a  survey  was  conducted  of  all  six  FSAs  to  capture  the  perspectives  of 
managers,  SEPG  members,  practitioners,  and  others  involved.  Because  all  FSAs  reported 
directly  to  the  FSO  prior  to  April  1998,  the  FSO  was  highly  involved  in  the  level  2  efforts 
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of  all  FS  As.  Additionally,  some  aspects  of  the  FSA’s  SPI  program  were  mandated  by  the 
FSO. 

The  review,  compiled  in  December  1997,  consists  of  sixteen  questions.  Ten  of  the 
questions  ask  respondents  to  provide  a  1-5  rating.  The  other  questions  are  open-ended 
and  comments  are  recorded.  Eight  of  the  questions  (6  rating  questions  and  2  open-ended) 
covering  concepts  most  relevant  to  this  thesis  are  included  in  this  section.  The  full  text  of 
the  review  is  provided  in  Appendix  B.  Data  from  the  review  are  analyzed  comparing 
FSA-KC  and  FSO  as  a  whole. 

Table  4.8  lists  six  questions  concerning  support  mechanisms  for  software  process 
improvement  and  the  ratings  provided  by  members  of  FSA-KC  and  by  the  members  of 
the  other  FSAs.  Results  of  all  FSAs  (including  FSA-KC)  are  averaged  and  listed  under 
the  heading  of  FSO.  For  all  questions  using  the  numeric  scale  a  rating  of  3.00  represents 
the  middle  ground  of  the  response.  Any  rating  below  3.00  indicates  a. disagreement  or 
low  appraisal,  while  any  rating  above  3.(K)  represents  agreement  or  a  high  rating. 
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FSA-KC 

FSO 

(N  =  50) 

(N  =237) 

Rate  senior  management  support  at  your  FSA  for  SPI. 

3.63 

3.70 

Rate  the  info,  related  to  improvement  activities  that  was  made 

available  to  you. 

3.61 

3.52 

Rate  the  training  you  received  on  SPI  initiatives. 

3.54 

3.40 

Rate  the  FSO  support  activities  to  SPI  initiatives  (These  activities 
include  training,  FSO  SPI  conferences,  SMS  workshops, 
and  process  definition). 

3.12 

3.07 

Rate  the  tools  provided  by  FSO  for  SPI  initiatives. 

2.70 

2.70 

Rate  the  local  SEPG  support  you  received  on  SPI  Initiatives. 

4.07 

3.62 

Table  4.8  FSO  Level  2  Post  Implementation  Review 


As  shown  in  Table  4.8,  support  mechanisms  internal  to  the  FSAs,  senior 
management  support,  SPI  related  information  made  available  and  training  on  SPI 
initiatives,  received  ratings  above  the  midpoint  (3.0).  SEPG  support  for  SPI  initiatives 
was  fated  high  FSO- wide,  with  FSA-KC  receiving  a  rating  higher  than  the  average  rating 
of  SEPGs  within  FSO.  In  contrast,  FS04evel  support  activities  received  lower  ratings 
than  local  support;  and  tools  provided  by  FSO  for  SPI  initiatives  received  the  worst  rating 
of  the  support  mechanisms. 

The  responses  to  open-ended  questions  shed  some  light  to  the  low  ratings  of  FSO 

support.  Some  respondents  commented  that  the  major  problem  with  FSO’s  approach  to 

employing  support  tools  is  that  tools  are  selected  based  on  their  success  within  a 

particular  automated  information  system  and  are  then  expected  to  yield  similar  success  in 

other  automated  information  systems.  The  following  worker  comments  regarding  the  use 

of  software  development  tools  mandated  for  use  by  FSO  are  illustrative  of  the  problem: 

AISs  are  expected  to  adopt  to  tools  developed  for  AISs  in  different  environments 
(i.e.  CMIS).  Trying  to  force  round  pegs  in  square  holes.  Attempting  to  use  LRSfor 
metrics  in  ways  it  is  inadequate  for. 
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They  all  have  that  awkward,  home-grown  feel.  Probably  would  have  been  better 
to  find  tools  that  suit  the  needs  rather  than  enforcing  tools  to  fit  needs  that  were 
never  envisioned  when  they  were  developed. 

CMIS  is  old  and  outdated.  Written  by  individuals  in  a  specific  site  and  everyone  is 
expected  to  use. 

Comments  and  responses  to  questions  regarding  FSO  support  mechanisms  have 
implications  on  the  responsiveness  of  externally  provided  support  and  on  technology 
insertion  FSO-wide.  The  implications  are  that  internal  support  mechanisms  are  more 
responsive  to  process  improvement  requirements  than  FSO  provided  support 
mechanisms,  which  are  expected  to  support  the  needs  of  all  six  FSAs.  This  is  particularly 
significant  as  it  applies  to  software  management  tools. 

Another  problematic  aspect  of  the  level  2  effort  identified  in  the  survey  was  the 
impact  of  process  improvement  on  customer  relations.  When  provided  the  open-ended 
statement:  The  most  significant  negative  impact  caused  by  SPI  efforts  was...’  the 
predominant  theme  was  that  process  improvement  activities  add  to  cost  (in  time  and 
money)  and  stress  customer  relations.  The  following  comments  are  representative  of  the 
theme. 

Need  more  customer  commitment  and  support  for  SPI.  The  program  is  too 
costly  and  important  to  FSO  not  to  gain  strong,  active,  early  commitment 
starting  at  the  top  of  the  DFAS  management  chain. 

Cost  too  high,  cost  to  implement  and  maintain. 

Added  paperwork  that  needs  to  be  charged  to  a  customer  who  already  doesn't 
have  enough  money  to  get  the  job  done. 

Customer  has  claimed  slower  turnaround  for  impacts  and  releases. 

These  responses  reinforce  the  data  gathered  in  the  FSA-KC  process  improvement 
survey  (section  B)  that  point  to  competition  for  resources  and  customer  support  as  the 
primary  challenges  to  the  organization’s  process  improvement  efforts.  The  implications 
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are  that  customer  support  may  not  be  sufficiently  addressed  by  the  leadership  of  FSA. 

The  expense  of  process  improvement  activities  in  time,  resources,  and  persoimel  requires 
not  only  internal  support  and  buy-in,  but  also  an  enlightened,  supportive  customer. 

D.  PERSONAL  INTERVIEWS 

1.  Management  Perspectives 

Semi-structured  interviews  were  conducted  during  both  visits  to  FSA-KC.  Due  to 
time  constraints  and  limited  personnel  availability,  only  managers  (Project  Officers, 
Division  Heads  and  Branch  Heads)  were  interviewed.  Management  perspectives  are 
provided  in  question  and  answer  format  in  this  section.  The  responses  to  these  questions 
are  indicators  of  management  expectations  for  process  improvement  efforts.  Director 
support  for  process  improvement,  organizational  goals,  and  areas  for  improvement  within 
the  organization.  The  questions  here  represent  a  subset  of  the  interview  questions.  The 
full  set  of  interview  questions  appear  in  Appendix  C. 

a.  Expectations  for  Process  Improvement  Efforts 

•  What  concrete  observable  outcomes  will  result  from  process  improvement 
efforts? 

•  Do  the  proposed  changes  make  the  employees  jobs  easier  of  harder? 

The  following  statements  are  representative  of  the  responses  to  these 

questions. 

More  structure  will  facilitate  control,  accountability,  and  efficiency. 

Better  quality.  Less  Production  Incident  Reports. 

Reduced  duplication  of  effort.  That  was  one  of  the  benefits  from  level  2. 

The  responses  echo  those  found  in  the  FSA-KC  process  improvement 
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survey  and  the  Level  2  Post  Implementation  Review.  Managers  are  receptive  to  the 
concept  of  software  process  improvement  and  have  positive  expectations.  All 
respondents  believe  that  planned  changes  will  make  employees  jobs  easier. 

b.  Director  Support 

•  What  does  the  Director  say  about  SPl?  What  does  he  do? 

Managers  cited  inconsistent  Director  support  for  SPI.  The  following 
statements  are  representative  of  the  responses  to  this  question. 

The  Director  talks  SPI,  but  does  not  provide  adequate  support. 

He  (the  Director)  wants  to  achieve  level  3...  he's  not  really  willing  to  make 

sacrifices  to  get  there.  His  #1  priority  is  to  get  the  release  out  on  time. 

The  Director  must  be  held  accountable  for  SPI  if  you  want  real  support  from  him. 

The  responses  here  are  an  indicator  of  competition  for  resources  and 
conflicting  priorities.  In  discussions  with  respondents  the  perception  is  that  the  Director 
does  in  fact  value  SPI  activities.  However,  within  the  constraints  of  limited  resources  and 
competing  priorities  SPI  initiatives  often  take  a  back  seat. 

c.  Organizational  Goals 

•  The  most  important  thing  to  the  leadership  of  FS A  is . . . 

The  question  of  what  is  the  most  important  thing  to  the  leadership  of  FSA, 
is  designed  to  assess  management’s  perceptions  of  organizational  goals.  This  is 
significant  because  if  the  organization’s  SPI  efforts  are  to  be  adopted  and  maintain 
consistent  support,  they  must  be  compatible  with  the  organi^tion’s  goals. 

Decreasing  cost  and  meeting  the  release  schedule  are  seen  as  the  primary 
goals  of  FSA.  All  responses  to  the  question  of  referenced  cost,  schedule  or  both.  The 
obsession  with  reducing  cost  appears  to  be  due  mostly  to  the  fee-for-service  concept  that 
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the  FS  A  works  under.  The  concept  of  fee-for-service  is  a  major  part  of  the  Department  of 
Defense's  attack  on  waste  and  abuse  of  government  funds,  by  encouraging  DoD  activities 
to  operate  like  commercial  businesses.  It  provides  visibility  of  fees  charged  by  service 
providers  (unit  costs),  with  the  ultimate  goal  of  eliminating  any  non-value  added  aspects 
of  business.  Decreasing  unit  costs  is  an  obvious  priority  of  any  fee-for-service  activity. 

d.  Areas  for  Improvement 

•  What  specific  parts  of  the  organization  provide  the  most  opportunity  for 
increases  in  efficiency  and  quality? 

This  question  is  designed  to  elicit  responses  that  address  problematic 

aspects  or  areas  for  improvement  within  the  organization.  Managers  see  the  issues  of 

customer-relations  and  requirements  analysis  as  the  are£^  with  the  highest  potential  to 

increase  the  organization's  efficiency  and  quality.  The  following  statements  are 

representative  of  the  responses  to  this  question: 

The  customer  has  set  unrealistic  requirements  and  schedule  demands...  SPI  is 
seen  as  delaying  the  process. 

They  (senior  management)  want  it  all-cost,  quality,  schedule— even  in  situations 
where  it  is  obvious  that  there  are  tradeoffs  to  be  made.  This  is  driven  mainly  by 
the  customer. 

This  is  another  reference  to  the  importance  of  the  customer  in  the 
organization’s  plans  and  practices.  As  Pasmore  (1988)  asserts,  the  external  environment 
(represented  here  by  the  customer)  is  the  ultimate  judge  of  an  organization’s  survival. 

The  comments  are  also  consistent  with  the  Level  2  Post  Implementation  Review 
comments  discussed  previously  (section  C).  This  demonstrates  the  problem  of  customer 
support  for  SPI  is  systemic  and  not  particular  to  FSA-KC. 
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2.  Customer  Perspectives 

Two  customer  representatives  for  the  MCTFS  AIS  were  interviewed  to  assess  the 
level  of  awareness,  buy-in  and  support  for  process  improvement.  They  were  asked  the 
following  questions: 

•  Have  you  been  briefed  on  SPI? 

•  What  are  your  initial  impressions  of  SPI  at  FSA? 

•  Will  SPI  initiatives  help  you?  Are  you  willing  to  pay? 

Both  interviewees  indicated  that  they  were  not  provided  a  briefing  on  SPI.  Neither 
could  pinpoint  how  FSA’s  level  2  efforts  impacted  the  customer  whom  they  represent. 
One  representative  indicated  that  FSAs  level  2  efforts  were  transparent  to  the  Marine 
Corps.  The  second  representative  remarked  "DFAS  headquarters  says  they  have  to  be  at 
‘a  level’.  It  would  be  easier  to  justify  the  investment  if  it  lowered  costs  or  increased 
quality  or  something". 

E.  GENERAL  IMPRESSIONS  FROM  SITE  VISITS 

During  the  initial  visit  to  FS  A-KC  the  author  had  the  opportunity  to  attend  a 
number  of  organizational  meetings  including:  The  SPI/SEPG  Meeting  between  Director, 
Deputy  Directors  and  SEPG;  MCTFS  Project  Officers  Meeting;  SEPG  Morning  Meeting; 
and  SEPG/FSO  Conference  Calls.  Discussions  in  these  meetings  provide  perspectives  on 
the  inner-workings  of  the  FSA  and  its  SPI  efforts. 

In  terms  of  the  organization's  readiness  for  software  process  improvement  efforts, 
indications  are  that  at  the  middle  management  level  and  above  there  is  enthusiasm  for 
SPI.  The  indications  are  that  SPI  initiatives  fit  into  the  current  organizational  practices. 
Specifically  managers  seem  well  aware  of  the  benefits  of  mature  software  engineering 
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practices  and  believe  SPI  activities  will  enhance  day  to  day  operations.  However, 
managers  are  equally  aware  of  the  constraints  of  SPI  implementation-limited  resources, 
fixed  schedules  and  lack  of  customer  support.  This  is  consistent  with  data  from 
interviews  and  archival  material. 

A  high  level  of  cooperation  exists  between  the  SEPG  and  AIS  managers.  SEPG 
members  are  able  to  use  technical  knowledge,  experience,  and  relationships  gained  in 
part  from  previous  work  within  the  divisions  to  generate  workable  action  plans  that 
reflect  the  input  of  personnel.  The  SEPG  is  viewed  favorably  as  internal  technical 
consultants  on  SPI  issues.  This  fact  indicates  possible  opportunities  for  the  organization 
to  enhance  its  process  improvement  efforts  by  expanding  the  role  of  the  SEPG. 

While  process  improvement  efforts  are  somewhat  compatible  with  organizational 
practices  they  are  separate  and  distinct  from  the  organization’s  day  to  day  goals.  Process 
improvement  efforts  have  a  loose  connection  to  the  organization’s  business  goals  but  they 
are  challenged  by  schedule  pressure,  other  priorities,  and  competition  for  resources. 
Support  for  SPI  from  the  Director  ebbs  and  flows  with  these  factors.  Recent  delays  in 
implementation  of  process  improvement  activities  in  the  Standard  Accounting  and 
Budgeting  Reporting  System  (S  ABRS),  due  to  a  backlog  of  key  modifications  requested 
by  the  customer,  are  an  example  of  how  intense  schedule  pressure  can  derail  SPI  efforts. 
While  the  Director  is  knowledgeable  and  enthusiastic  about  the  potential  benefits  of  SPI, 
he  appears  to  be  constrained  by  environmental  pressures  which  many  times  conflict  with 
SPI  efforts.  This  is  consistent  with  data  obtained  from  personal  interviews  and  the  FSA- 
KC  survey. 
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There  is  a  perceived  emphasis  on  “making  the  grade”  in  the  process  improvement 
initiative.  This  detracts  from  key  objectives  sought  through  process  improvement: 
improving  product  quality,  and  improving  customer  satisfaction.  The  “grade”  aspect  is 
the  explicit  or  implicit  view  that  the  primary  goal  of  SPI  is  the  attainment  of  a 
certification.  This  is  evident  in  directive  statements  like  “we  will  be  at  level  3  by  October 
1999.”  This  approach  is  particularly  detrimental  when  the  date  is  selected  without  an 
appraisal  of  the  social  and  technical  issues  that  must  occur  to  accomplish  the  goal. 
(Pasmore,  1988)  This  “grade”  centered  approach  to  process  improvement  strains  the 
alignment  of  SPI  with  organizational  practices  and  goals. 

While  the  reward  system  at  FSA  does  not  explicitly  address  software  process 
improvement,  it  is  flexible  enough  to  be  used  as  a  tool  to  reinforce  desired  behaviors. 
During  and  subsequent  to  MCTFS’  level  2  efforts,  senior  management  rewarded  a 
number  of  individuals  for  their  process  improvement  efforts.  This  provides  additional 
confirmation  of  the  survey  data  reporting  some  use  of  rewards.  It  may  also  suggest  that 
this  use  of  rewards  is  not  sufficiently  publicized,  thus  explaining  the  low  agreement  on 
the  use  of  rewards  reported  in  the  survey  (see  Table  4.6). 

F.  SUMMARY  AND  IMPLICATIONS 

In  Chapter  H,  best  practices  were  derived  based  on  a  review  of  relevant  change 
management  theory  and  software  process  improvement  technical  reports  and  papers. 
These  best  practices  were  categorized  as  concepts  related  to:- 1)  preparing  the 
organization  for  process  improvement,  2)  implementing  process  improvements,  and  3) 
sustaining  the  improvement  effort.  This  section  summarizes  the  results  of  the  data 
analyzed  and  examines  the  implications  of  the  results  along  the  same  criteria. 
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1.  Preparing  the  Organization  for  Process  Improvement 

Concepts  relative  to  preparing  the  organization  for  process  improvement  include  a 
clear  understanding  of  current  organizational  goals,  communication  of  a  clear  purpose  for 
process  improvement,  establishing  compatibility  with  organizational  practices  and  goals 
and  ensuring  top  management  support.  Dalziel  and  Schoonover  (1988)  discuss 
compatibility  of  the  planned  change  with  organizational  goals  as  a  primary  factor  in 
getting  an  organization  ready  for  a  planned  change.  A  good  starting  point  is  ensuring  that 
current  goals  are  understood  and  shared  by  all.  The  goals  of  the  Director  FSA  as  stated  in 
the  October,  1997  document  entitled  FSA  Director's  Goals  are:  to  provide  exemplary 
customer  service,  reduce  business  costs,  continue  process  improvement  to  reach  level  3 
of  the  CMM,  and  develop  a  migration  strategy  to  implement  the  Joint  Technical 
Architecture.  (FSA,  1997)  In  the  document  the  Director  commits  to  using  customer 
service  as  the  driving  force  for  FSA  efforts. 

Results  of  the  data  indicate  that  cost,  schedule,  and  customer  service  are  all 
viewed  as  measures  of  organizational  success.  It  is  significant  that  no  single  factor  really 
emerges  as  a  primary  measure  of  the  organization's  success.  In  the  software  industry 
there  are  constant  tradeoffs  between  customer  demands,  cost,  schedule,  and  product 
quality.  The  perception  on  the  part  of  managers  and  practitioners  that  senior  management 
does  not  recognize  that  there  are  tradeoffs  to  be  made  can  lead  to  frustration  and  hostility. 
As  one  interviewee  put  it:  "They  want  it  all-cost,  schedule,  quality— even  when  they 
know  it's  impossible." 

It  is  also  significant  that  product  quality  is  not  one  of  the  Director's  explicit 
primary  goals.  Product  quality  is  addressed  within  the  context  of  the  other  goals. 
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However,  the  data  in  this  thesis  suggest  that  employees  believe  that,  while  product 
quality  is  the  primly  measure  of  success  in  their  work  areas,  it  is  subordinate  to  cost  and 
schedule  at  FSA.  This  attitude  about  product  quality  is  not  consistent  with  the  attitude 
that  is  required  to  sustain  a  process  improvement  effort.  It  does  not  harness  the  employee 
motivation  for  producing  quality  software  that  the  data  indicate  is  present  at  FSA. 
Additionally  the  discontinuity  between  employee  perceptions  of  measures  of  success  in 
their  work  place  and  organizational  measures  of  success  may  suggest  that  employees  do 
not  understand  how  their  efforts  contribute  to  the  organization’s  success. 

Bridges  (1988)  asserts  the  importance  of  a  clear  purpose  for  a  planned  change. 
The  purpose  should  align  with  internal  organizational  characteristics.  In  Chapter  I, 
software  process  improvement  was  defined  as  those  practices  and  procedures  aimed  at 
facilitating  the  consistent  and  predictable  production  of  quality  software.  However,  the 
purpose  of  FSA's  SPI  efforts  is  not  as  clearly  defined.  Employees  are  more  apt  to  believe 
that  the  purpose  of  FSA's  process  improvement  efforts  are  to  satisfy  a  requirement  from 
higher  headquarters  or  DoD  as  opposed  to  increasing  software  quality.  This  dilutes 
employee  commitment  to  the  effort  for  at  least  two  reasons:  1)  they  don't  really  know 
how  their  efforts  contribute  to  the  organization’s  process  improvement  efforts  and  2) 
some  may  perceive  that  the  process  improvement  effort  is  of  limited  duration,  and  they 
can  endure  until  it  passes.  To  the  extent  that  employees  view  SPI  as  mandated  from 
higher  authority  without  acknowledging  the  potential  benefits  to  the  organization’s 
performance,  there  will  likely  be  greater  resistance  to  changes  involved  in  SPI 
implementation. 
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Dalziel  and  Schoonover  (1988)  believe  that  every  leader  who  initiates  a  policy, 
program,  or  practice  should  assess  the  compatibility  of  the  initiative  with  the  current 
organizational  practices.  The  mismatch  between  a  change  initiative  and  an  organization's  . 
daily  practices  is  a  key  source  of  resistance  to  change.  The  data  in  this  thesis  indicates 
that  employees  believe  process  improvement  efforts  are  somewhat  compatible  with  their 
current  practices.  More  report  compatibility  than  not,  but  a  substantial  minority  report  a 
difficult  balance  between  SPI  and  day  to  day  work  or  see  neglect  of  important  areas  as  a 
result  of  SPI.  This  concern  reinforces  the  question  of  the  importance  of  SPI  goals  relative 
to  the  existing  performance  goals  of  the  organization.  The  majority  of  respondents 
believe  SPI  initiatives  will  make  their  jobs  easier.  Previous  success  in  implementing  level 
2  process  improvements  in  the  MCTFS  AIS  is  in  large  part  responsible  for  the  favorable 
perception  of  the  benefits  of  process  improvement.  The  indication  is  that  resistance  to 
process  improvement  efforts  is  more  a  function  of  SPI  program  management  issues  than 
technical  issues. 

Support  of  top  management  is  critical  at  the  initial  stages  and  throughout  the 
process  of  planned  change.  (Dalziel  &  Schoonover,  1988)  To  reinforce  the  new  change, 
senior  management  must  send  consistent  messages  about  the  importance  and  priority  of 
the  change.  (Bridges,  1991)  The  data  indicate  that  members  of  FSA  are  receiving  mixed 
messages  about  the  priority  of  SPI.  In  discussions  with  respondents,  the  perception  is  that 
the  concept  of  improving  software  processes  is  important  to  the  Director.  ,  However, 
within  the  constraints  of  limited  resources,  competing  priorities,  and  customer  demands 
SPI  initiatives  are  often  subordinate.  The  data  support  the  fact  that  employees  are  not  sure 
that  SPI  is  a  high  priority  to  FSA.  It  is  recognized  by  most  in  the  organization  that  the 
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Director's  support  for  SPI  is  constrained  by  pressures  from  the  customer  and  higher 
headquarters.  Also  the  low  level  of  support  and  buy-in  for  process  improvement  at  the 
practitioner  level  indicates  that  practitioners  are  not  sufficiently  involved  in  SPI  activities. , 

2.  Implementing  Process  Improvements 

Internal  and  external  support  mechanisms  were  examined  in  this  chapter.  The 
results  indicate  that  internal  support-senior  management  support,  SEPGs,  information 
and  training— is  more  responsive  to  organizational  needs  than  external  support 
mechanisms  provided  by  the  FSO.  The  costs  of  standardization  of  tools  and  procedures 
may  outweigh  the  benefits.  It  is  logical  that  a  smaller  defined  organization  (i.e.,  a  single 
FSA  as  opposed  to  all  FSAs)  allows  for  more  focused  and  responsive  process  refinements 
than  a  larger  organization. 

The  issue  of  software  development  tools  mandated  for  use  by  the  FSO  was 
particularly  problematic  and  has  implications  for  technology  insertion.  The  insertion  of 
technology  in  the  work  environment  whether  it  is  in  the  form  of  process  improvements  or 
support  tools  is  important  enough  to  warrant  evaluation  and  review.  All  of  the  successful 
case  studies  reported  in  Chapter  II  indicate  that  piloting  of  processes  and  tools  is  a  key 
aspect  of  their  SPI  efforts.  Pilot  studies  of  specific  techniques  and  technologies,  and 
subsequent  evaluation  of  the  impact  of  the  techniques  and  technologies  prior  to  their 
widespread  implementation  will  save  time  and  resources  in  process  changes  that  are  not 
effective.  The  case  studies  also  reported  that  tailoring  process  improvement  mechanisms 
to  locally  unique  requirements  makes  for  more  appropriate  solutions.  Tailoring  and 
piloting  may  decrease  the  cynicism  that  results  when  employees  perceive  that  they  are 
being  forced  to  use  tools  that  are  not  responsive  to  their  environment. 
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The  data  from  the  FS  A-KC  survey  and  the  FSO  Level  2  Post  Implementation 
Review  indicate  that  the  SEPG  is  viewed  favorably  within  FSA-KC.  This  fact  offers 
possibilities  for  expanding  the  role  of  the  SEPG  to  enhance  the  achievement  of  process 
improvement  goals  as  well  as  other  organizational  goals.  The  added  exposure  of  the 
SEPG  may  facilitate  communication  of  process  improvement  goals  and  organizational 
buy-in.  Especially  at  the  practitioner  level  where  the  data  suggests  there  is  a  need  for 
more  employee  involvement  and  participation. 

3.  Sustaining  the  Process  Improvement  Effort 

Sustaining  process  improvement  efforts  is  a  function  of  encouraging  innovation, 
instilling  a  long-term  focus  on  process  improvement,  and  managing  the  environment. 

An  organization’s  ability  to  improve  is  a  function  of  the  leadership  commitment  to  create 
a  culture  that  invites  innovation  and  continuous  learning.  (Quann,  1995)  As  Bridges 
(1991)  states,  people  naturally  generate  solutions  to  problems  they’ve  been  living  with. 
What  they  seldom  do  without  encouragement  and  support  is  try  their  ideas.  The  data  in 
the  FSA-KC  process  improvement  survey  indicate  that  most  employees  believe  that  their 
ideas  and  recommendations  for  process  improvement  will  be  acknowledged  and  acted 
upon.  However,  the  majority  doesn’t  believe  that  they  will  be  rewarded  for  innovative 
ideas.  This  may  be  remedied  by  more  explicitly  linking  rewards  to  process  improvement 
efforts  and  making  the  rewards  more  public. 

Continuous  software  process  improvement  is  a  long-term  undertaking.  The  results 
in  this  chapter  indicate  that  FSA  does  not  approach  SPI  with  a  long-term  focus.  An 
approach  to  process  improvement  that  is  centered  on  reaching  a  certain  level  ('making  the 
grade')  within  a  fixed  timeline  can  be  counterproductive  for  a  number  of  reasons.  First, 
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'making  the  grade'  is  not  a  goal  that  aligns  with  other  organizational  goals.  Consequently, 
it  does  not  create  sufficient  buy-in  for  process  improvement  activities  throughout  the 
organization.  Additionally,  trying  to  force  change  into  a  prefixed  timeline  suggests  that 
the  rate  of  change  is  not  geared  to  organizational  factors.  (Dalziel  &  Schoonover,  1988) 
As  Pasmore  (1988)  asserts,  the  external  environment  is  the  ultimate  judge  of  an 
organization's  survival.  An  overwhelming  theme  of  the  results  from  this  study  is  that 
managing  customer  expectations  and  ensuring  customer  support  are  the  problematic  areas 
of  FSA's  SPI  effort.  Specifically,  the  results  indicate  that  the  customer  is  not  educated  on 
process  improvement  and  its  benefits.  Customer  demands  represent  constraints  and 
opportunities  for  the  organizations  that  serve  them.  Successful  organizations  do  more 
than  react  to  the  environment;  they  take  steps  to  transform  the  environment  itself  to  make 
it  more  conducive  to  their  well  being.  (Pasmore,  1988)  FSA's  process  improvement 
efforts  are  financed  to  a  large  extent  by  its  customers.  Therefore  it  is  critical  that  the 
customer  be  educated  on  the  benefits  of  process  improvement. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 

The  intent  of  this  thesis  is  to  facilitate  the  software  process  improvement  efforts 
of  FS  A-KC  and  to  generalize  the  knowledge  developed  through  this  research  to  other  SPI 
projects.  In  this  chapter,  the  high  points  from  the  preceding  chapters  are  first  summarized. 
This  is  followed  by  discussion  of  the  primary  conclusions  of  this  thesis  and  sets  of 
recommendations:  1)  for  FSA,  2)  for  other  SPI  projects  and  3)  for  further  research. 

A.  SUMMARY 

Chapter  I  provides  background  information  on  the  state  of  software  engineering 
today.  Specifically,  software  engineering  as  a  discipline  lags  behind  other  engineering 
disciplines  in  terms  of  the  software  industry's  ability  to  produce  quality  software  within 
cost  and  schedule  constraints.  Chapter  I  also  discusses  the  challenges  to  implementing 
software  process  improvement  initiatives.  Lack  of  compatibility  of  SPI  with 
organizational  practices  and  goals,  complexity  of  SPI  concepts,  and  lack  of  consistent 
buy-in  and  support  for  SPI  are  among  the  primary  challenges  to  a  SPI  effort. 

Chapter  II  reviews  three  theoretical  perspectives  on  organizational  change.  The 
work  of  Dalziel  and  Schoonover  (1988),  Pasmore  (1988),  andBridges  (1991)  provide  the 
frameworks  for  analysis  in  this  study.  Although  their  approaches  differ,  there  is  some 
overlap  in  their  coverage  of  certain  concepts.  The  overarching  themes  of  the  literature 
addressed  assessing  an  organization's  readiness  for  change,  change  implementation 
strategies,  and  sustaining  change.  Using  this  framework  the  software  process 
improvement  efforts  of  four  organizations  are  reviewed.  In  many  cases  the  success  of  the 
SPI  efforts  within  these  organizations  can  be  attributed  to  the  practical  application  of  one 
or  more  of  the  change  concepts  reviewed.  The  change  theory  and  the  actual 
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implementation  strategies  of  these  organizations  lead  to  a  set  of  best  practices.  These  best 
practices  are  categorized  as  concepts  related  to: 

•  Preparing  the  Organization  for  Process  Improvement 

•  Implementing  Process  Improvements 

•  Sustaining  the  Process  Improvement  Effort 

Strategies  that  prepare  the  organization  for  process  improvements  include 
examining  attributes  of  organizational  readiness  (history  of  change,  clarity  of 
expectations,  origin  of  idea  or  problem,  support  of  top  management,  and  compatibility 
with  organizational  goals)  and  scanning  the  environment  to  assess  the  organization’s 
preparation  for  change  and  understand  the  overall  context  of  the  process  improvement 
effort. 

Effective  implementation  of  process  improvements  typically  include  deliberate 
design  for  planned  change,  the  use  of  transition  management  teams,  treating  SPI  as  a 
project,  and  maximizing  involvement  in  the  SPI  program.  These  measures  are  designed  to 
facilitate  the  acceptance  of  process  improvement  efforts  by  communicating  a  purpose  and 
,  closely  linking  process  improvements  to  organizational  practices  and  goals.  Additionally, 
these  measures  enhance  participation  at  all  levels  of  the  organization  to  increase  buy-in 
and  support. 

Concepts  related  to  sustaining  the  process  improvement  effort  include  piloting, 
integrating  process  improvements  within  the  project  cycle,  and  personnel  training  in 
concepts  of  change  management.  The  goal  of  these  measures  is  to  create  a  learning 
organization. 
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In  Chapter  III,  the  research  methods  are  discussed,  including  a  process 
improvement  survey,  archival  material,  personal  interviews,  and  site  visits  that  were  used 
to  assess  the  challenges  to  FSA's  process  improvement  efforts. 

Chapter  IV  provides  results  and  analysis  of  the  data  obtained  from  the 
aforementioned  sources.  The  results  are  analyzed  and  examined  within  the  framework  of 
the  best  practices  discussed  earlier.  The  results  suggest  that  senior  management  at  FSA 
could  do  more  to  prepare  the  organization  for  process  improvement  efforts.  Specifically, 
clarifying  current  organizational  goals,  communicating  a  clear  purpose  for  SPI  and 
aligning  SPI  initiatives  with  organizational  practices  and  goals  are  areas  that  need  to  be 
addressed. 

The  results  also  indicate  that  internal  support  mechanisms — senior  management 
support,  SEPGs,  local  information  and  training — are  more  responsive  to  organizational 
needs  than  external  support  provided  by  the  FSO.  The  FSO’s  approach  to  employing 
technical  support  tools  is  particularly  problematic.  Employees  expressed  concern  that  the 
tools  mandated  by  FSO  do  not  address  unique  characteristics  of  their  particular 
automated  information  systems. 

The  results  suggest  that  there  are  mechanisms  in  place  to  encourage  innovation. 
However,  they  may  not  well  publicized.  The  results  suggest  there  is  a  need  for  greater 
involvement  in  SPI  at  the  section  level. 

Finally,  the  results  suggest  that  FSA  is  not  approaching  SPI  with  a  long-term 
focus.  The  emphasis  on  quickly  making  the  grade  is  counter  to  the  continuous  process 
improvement  paradigm.  Additionally,  by  not  educating  and  involving  the  customer  in  SPI 
activities,  FSA  is  not  addressing  a  significant  source  of  resistance  to  SPI. 
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A  summary  of  the  challenges  to  the  FSA's  process  improvement  initiative  as 
presented  in  Chapter  IV  follows: 

•  Lack  of  clarity  and  consistency  of  goals  throughout  the  organization. 

•  Software  process  improvement  activities  are  not  aligned  with  organizational 
goals. 

•  External  support  mechanisms — specifically  FSO  provided  software 
development  tools-do  not  enhance  software  process  improvement  efforts. 

•  Lack  of  consistent  senior  management  support  for  software  process 
improvement. 

•  Lack  of  customer  buy-in  and  support  for  software  process  improvement 
initiatives. 

B.  CONCLUSIONS  FOR  FSA-KC 

Balancing  customer  demands,  competing  priorities  and  limited  resources  is  a  part 
of  everyday  life  for  the  senior  management  of  FSA.  However,  implementing  SPI  adds 
resistance  to  change,  transition  management,  technology  insertion,  and  other  social  and 
technical  issues  to  the  challenges.  Past  success  in  attaining  CMM  level  2,  a  qualified  and 
respected  SEPG,  and  a  workforce  that  is  motivated  tO  produce  quality  products  are  assets 
in  the  FSA’s  drive  for  further  process  improvement. 

SPI  is  a  long-term  undertaking,  requiring  a  large  investment  in  time  and 
resources.  The  organization  undoubtedly  has  other  daunting  priorities — ^modifications  to 
existing  automated  information  systems,  designing  new  automated  information  systems, 
Y2K,  etc.  The  leadership  of  FSA  must  evaluate  their  plan  for  implementing  SPI  and  how 
it  fits  into  existing  organizational  goals,  priorities,  opportunities  and  constraints.  They 
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must  involve  all  key  stake-holders,  especially  customers,  in  the  planning.  By  providing  a 
well-communicated  vision  and  a  consistent  commitment  to  continuous  process 
improvement  over  the  long-term  they  can  better  focus  the  efforts  of  the  employees. 

Tom  DeMarco  (noted  author  and  consultant  on  managing  software  projects)  says 
competent  software  engineers  inherently  want  to  produce  quality  software,  it  is 
management’s  job  to  remove  the  obstacles. 

C.  RECOMMENDATIONS 

In  this  section  recommendations  are  organized  to  address  three  general  areas:  1) 
FSA-KC,  2)  other  SPI  projects  and  3)  further  research.  Each  is  addressed  in  turn. 

1.  Recommendations  for  FSA-KC 

The  following  best  practices  are  recoimnended  as  solutions  to  the  challenges 
facing  FSA-KC. . 

•  Explicit  Design  for  Plaiined  Change 

•  Transition  Management  Teams 

•  Piloting 

•  Integration  of  process  improvement  activities  into  the  project  cycle 

•  Scanning  the  environment 
These  practices  are  discussed  below. 

t 

a.  Change  Design 

The  nature  of  the  Defense  Finance  and  Accounting  Service  organization  is 
to  change,  and  to  do  so  frequently.  Its  mission  is  to  consolidate,  standardize,  and  integrate 
finance  and  accounting  procedures,  operations,  and  systems.  The  mission  guarantees  that 
at  all  levels  of  the  organization  there  will  be  changes  in  jobs,  procedures,  and  systems. 
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There  will  be  realignments  and  reorganizations.  Under  these  circumstances  there  are  two 
factors  which  are  critical  to  sustain  the  momentum  of  the  organization  during  times  of 
change.  The  first  is  a  solid  foundation  in  exactly  what  the  goals  of  the  organization  are. 
Clear  organizational  goals  provide  some  measure  or  normalcy  in  times  of  chaotic  change. 
A  firm  commitment  to  customer  service  and  product  quality  is  likely  to  survive  any 
changes  the  organization  undertakes. 

The  second  factor  is  an  explicit  design  for  planned  change.  It  is  of  critical 
importance  that  those  at  the  upper  levels  in  the  organization  be  conscious  of  the 
disruptive  nature  of  change,  of  the  transitions  necessary  to  successfully  change,  and  of 
the  value  of  information  during  times  of  change.  An  overall  change  design  that 
recognizes  and  addresses  the  dynamics  of  change,  and  make  senses  of  the  changes  within 
the  context  of  well-establishe(^  organizational  goals  can  be  of  value  to  FSA. 

The  change  design  should  include  specific  plans  for  communication,  work 
design,  piloting  efforts,  adjusting  reward  systems,  providing  appropriate  training,  and 
measuring  performance.  For  example,  to  enhance  communication  the  design  could 
include  mechanisms  for  providing  clarity  and  visibility  to  organizational  goals.  Possible 
vehicles  for  promoting  organizational  goals  are  vision  statements,  newsletters,  bulletin 
boards,  and  direct  correspondence  from  leaders  of  the  organization.  The  plan  for  work 
design  could  include  assigning  roles  and  responsibility  for  certain  aspects  of  the  SPI 
effort.  While  these  items  are  common  in  most  organizations,  they  may  not  be  used  to 
their  maximum  potential.  The  use  of  multiple  mechanisms  with  a  consistent  theme 
increases  the  chances  that  the  theme  will  be  communicated  throughout  the  organization. 
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Emerging  changes  in  the  organization's  functions,  procedures,  and  operations 
should  be  related  to  established  organizational  goals.  The  leadership  of  the  organization 
should  group  related  changes  where  possible,  and  eliminate  or  minimize  the  impact  of 
unrelated  changes.  When  there  are  simultaneously  occurring  change  initiatives  the 
leadership  of  the  organization  should  address  any  apparent  conflict  between  the  change 
initiatives  and  between  the  changes  and  the  organization's  goals.  This  will  mitigate  the 
confusion  on  the  part  of  the  employees  of  balancing  the  new  initiative  with  existing 
practices  and  goals. 

b.  Transition  Management  Teams 

As  detailed  in  chapter  II,  transition  management  teams  (TMTs),  represent 
one  strategy  used  effectively  in  best  practices  as  part  of  a  planned  change  design.  TMT’s 
consisting  of  personnel  representing  all  levels  of  the  organization,  provide  a  feedback 
mechanism  for  assessing  the  "real  impact"  of  improvement  activities,  demonstrate 
organizational  concern  for  the  people,  provide  linkage  across  organizational  levels,  and 
assign  responsibility  for  change  implementation. 

FSA  should  leverage  the  trust  and  confidence  that  the  organization  has  for 
the  SEPG  to  facilitate  the  TMT  function.  Data  in  Chapter  IV  indicate  possible 
deficiencies  in  employee  involvement  and  participation  at  the  practitioner  level.  Because 
the  SEPG  has  direct  and  frequent  communications  with  the  Director  they  are  ideal 
candidates  to  be  the  focal  point  of  the  transition  management  function.  SEPG  members 
could  lead  TMTs  in  various  parts  of  the  organization  to  facilitate  participation,  the  flow 
of  information,  and  feedback. 
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c.  Piloting 

Piloting  techniques  are  used  to  evaluate  new  technologies  and  methods  in 
test  situations  or  small  segments  of  the  organization  prior  to  their  implementation 
organization  wide.  Piloting  techniques  can  help  to  mitigate  the  problem  of  generic 
technical  support  tools  that  do  not  enhance  local  SPI  efforts,  for  example.  The  procedure 
involves  pilot  studies  of  specific  techniques  and  technologies  on  a  project,  analysis  of 
results,  and  evaluation  of  whether  the  techniques  or  technologies  are  appropriate  for  use 
organization  wide.  It  is  important  to  distinguish  the  organization  in  this  case  as  the  FSA 
and  not  the  FSO.  With  the  increasing  number  of  automated  information  systems  within 
each  FSA  it  is  very  challenging  to  design  software  tools  that  will  be  appropriate  for  all  six 
FSAs.  It  is  perhaps  more  feasible  for  each  FSA  to  design  its  own  software  tools  based  on 
the  unique  characteristics  of  its  automated  information  systems  and  local  best  practices. 
This  approach  makes  the  goals  and  characteristics  of  the  local  organization  the  driving 
element  of  change  (McGarry  et.  al,  1994)  and  will  be  better  received  by  organizational 
members. 

d.  Integration  of  SPI  Activities  into  the  Project  Cycle 

By  integrating  small  process  improvement  activities  within  the  project 
cycle,  FSA  can  more  closely  link  the  process  improvement  efforts  to  the  organization's 
business  goals.  This  process  can  be  reinforced  with  project-based  metrics  of  performance 
and  structured  reviews  of  process  improvement  activities.  Integrated  into  a  project,  the 
status  of  a  process  improvement  activity  can  be  evaluated  as  often  as  the  project. 
Additionally,  we  "piggy-back"  on  the  existing  reward  structures  and  motivations  geared 
toward  successful  project  releases.  The  primary  benefit  of  this  approach  is  that  it  keeps 


98 


software  process  improvement  visible.  It  keeps  senior  management  involved  in  the 
process  improvement  efforts.  This  approach  establishes  an  atmosphere  of  continuous 
process  improvement  as  opposed  to  bursts  of  SPI  activity  followed  by  lulls  which  send 
conflicting  message  about  the  importance  of  process  improvement. 
e.  Scanning  the  Environment 

Customer  ignorance  of  process  improvement  activities  has  the  most 
potential  for  derailing  FSA's  SPI  efforts.  Pasmore  (1988)  states  that  responses  to 
environmental  demands  can  take  one  of  two  forms.  Reacting  to  the  demands  as  they  are 
presented  or  transforming  the  environment  so  as  to  eliminate  or  alter  the  demands. 
Currently  FS  A  is  taking  the  former  approach.  The  customer  who  is  not  knowledgeable  of 
the  FSA’s  SPI  program  can  and  has  made  decisions  that  impact  the  organization’s  SPI 
efforts.  The  postponement  of  SPI  activities  within  the  SABRS  automated  information 
system  is  an  example.  The  problem  is,  it  is  would  be  of  most  benefit  to  the  organization 
to  take  the  latter  approach.  The  customer  provides  the  main  source  of  financing  for  SPI. 
Therefore,  it  is  logical  to  educate  the  customer  on  SPI. 

The  search  conference  and  open  systems  planning  methods  detailed  in  Chapter  n 
provide  techniques  for  understanding  and  assessing  the  demands  and  opportunities  that 
the  environment  presents  to  a  particular  change  initiative.  The  value  of  these  techniques 
is  that  it  forces  the  organization  to  address  customer  concerns  and  priorities  as  they  relate 
to  the  planned  initiative. 

It  seems  that  at  a  minimum,  the  organization  should  embark  on  a  campaign  to 
educate  the  customer  representatives  on  the  value  of  software  process  improvement.  A 
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more  ambitious  goal  is  the  education  of  the  real  customers — ^the  United  States  Marine 
Corps  and  the  Department  of  Defense — on  software  process  improvement. 

The  table  below  summarizes  the  challenges  and  recommended  solutions  for  the 
FSAs  software  process  improvement  initiatives. 


CHALLENGES 

RECOMMENDATIONS 

•Clarity  of  ' — 

w  •Change  Design 

•organizational  goals. 

•Alignment  of  SPI  with.^-^ 

•Transition  Management  Teams 

•organizatonal  goals. 

•Tftchnical  support  does 

- ►■•Piloting  and  Pathfinding 

•not  enhance  SPI. efforts. 

•Tnronsistent  senior 

- ^►•^•Integration  of  process  improvement 

•management  support. 

•activities  into  the  project  cycle. 

•  •Lack  of  customer  buy-in 

- 

•and  support. 

^  •Scanning  the  environment. 

Table  5.1  FSA  challenges  and  recommendations 
2.  Recommendations  for  Other  SPI  Projects 


It  is  likely  that  a  well-defined  process  will  help  most  software  development 
projects  to  succeed.  By  keying  on  the  unique  software  characteristics  of  the  local 
organization  process  improvement  efforts  will  more  likely  lead  to  product  improvements. 
Therefore  it  seems  that  the  key  challenge  in  implementing  software  process 
improvements  is  tailoring  the  effort  to  the  needs  of  the  organization. 

Software  process  improvement  is  a  process  of  organizational  change.  The  CMM 
does  not  address  issues  of  change  management.  Leaders  seeking  to  implement  SPI  should 
study  change  theory  because  ultimately  the  job  of  improving  processes  comes  down  to 
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changing  the  ways  that  people  do  things.  The  change  leader  must  understand  and 
manipulate  the  human  and  organizational  aspects  of  change. 

3.  Recommendations  for  Further  Study 

As  noted  in  Chapter  IV,  the  process  improvement  survey  was  administered 
without  the  questions  regarding  demographics.  It  was  the  intent  of  the  author  to  request 
demographic  data  that  indicates  organizational  level  (Senior  Management,  Management, 
or  Section  Personnel),  and  the  division  that  the  respondent  works  in.  Administering  the 
survey  with  the  benefit  of  demographic  data  will  allow  for  many  interesting  comparisons. 
For  example  the  results  could  be  analyzed  for  trends  and  gaps  across  organizational 
levels  and  across  the  divisions  within  FS  A.  Additionally  the  survey  could  be 
administered  at  different  FS  As  to  assess  the  issue  on  a  broader  scale. 
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APPENDIX  A.  PROCESS  IMPROVEMENT  SURVEY 


21  July  98 

DFAS-KC 

MEMORANDUM  FOR  DISTRIBUTION 
SUBJECT:  Software  Process  Improvement  Survey 

Attached  is  a  Software  Process  Improvement  Survey  that  each  FSA  employee  is  requested  to  complete. 
This  survey  contains  questions  about  the  software  process  improvement  initiative 
currently  taking  place  within  the  organization.  The  questions  ask  about  various  aspects  of 
the  organization  and  the  process  improvement  effort. 

Survey  results  will  be  analyzed  by  Captain  Wendell  Bazemore;  a  student  of  the  Naval 
Postgraduate  School,  for  the  purpose  of  facilitating  the  on-going  process  improvement 
efforts. 

Survey  responses  will  not  be  associated  with  any  employee.  Limited  demographic  data 
will  be  used  solely  to  develop  pooled  information  for  the  survey  categories. 


William  G.  Head 
Director  for  Information 
and  Technology 


Attachment: 
As  stated 
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Demographic  Data 


1.  Division: 

[]KT 

[]KD 

[]KA 

[]KS 

[]KM 

[]KI 

[]KB 

[]KE 

[]  Other _ 

2.  Role: 

[  ]  Senior  Management  (Project  Officers,  Division  Heads) 
[  ]  Management  (Branch  Heads) 

[  ]  Section  Personnel 

3.  Software  Experience _ yrs. 

4.  Years  at  IT&D _ 

5.  [  ]  Civilian  [  jMilitary 


.104 


1  .To  what  extent  Is  customer  satisfaction  used  as  a 
measure  of  success  at  FSA? 


Not  at  all  Very  Some  Very  Don’t 
Little  What  Much  Know 


Customer  Satisfaction 

Quality 

Cost 

Schedule 


[] 

n 

[] 

[] 


[] 

[] 

[] 

[] 


[] 

[] 

[] 

[] 


[] 

[] 

[] 

[] 


[] 

[] 

[] 

[] 


2.  To  what  extent  is  each  of  the  following  used  as  a  measure 
of  success  in  your  immediate  work  area? 

Customer  Satisfaction 

Quality 

Cost 

Schedule 


[] 

[] 

[] 

[] 


[] 

[] 

[] 

[] 


[] 

[] 

[] 

[] 


[] 

[] 

[] 

[} 


[] 

[] 

[] 

[] 


3.  Restructuring  initiatives  within  DFAS  have  had  a 
positive  affect  day  to  day  operations  at  FSA 


Strongly  Disagree  Agree  Strongly  Don’t 
Disagree  Agree  know 


[]  []  []  []  [] 


4.  Restructuring  initiatives  within  DFAS  have  had  a  positive 
affect  on  Software  Process  Improvement  efforts  at  FSA. 


[]  []  []  []  [] 


5.  Give  your  best  estimate  of 
the  organizations 
current  software  process 
maturity  level 


Level  1 .( Initial)  Few  process  are  defined.  Success  depends  on 
Individual  effort. 

^  Level  2  (Repeatable)  Project  management  processes  are  established. 
Realistic  project  commitments  are  based  on  previous  projects. 

Level  3  (Defined)  All  projects  make  use  of  an  organized,  documented, 
and  standardized  set  of  activities  that  are  institutionalized 
throughout  the  org^ization. 

Level  4  (Managed)  Detailed  metrics  are  collected  for  both  process  and 
quality,  and  used  to  quantitatively  manage  software  processes. 

Level  5  (Optimized)  Continuous  process  improvement  is  achieved 
through  metrics  and  feedback.  New  ideas  and  technologies  are 
constantly  tested. 


6.  In  your  judgement,  how  much  buy-in  and  support 
for  process  improvement  is  there  among: 


None 


Little 


Some 


Very 

Much 


Don’t 

Know 


Senior  Management 
Middle  Management 
Section  Personnel 


n 

[] 

[] 


[] 

[] 

[] 


[] 

[} 

[] 


n 

[] 

[] 


[] 

[] 

[] 


7.  Recommendations/ideas  for  process  improvement 
are  acknowledged  or  acted  on. 


Strongly  Disagree  Agree  Strongly  Don’t 
Disagree  Agree  know 
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[] 

[] 

[] 

[] 

[] 

8.  Successful  ideas  or  recommendations 
for  process  improvements  are  rewarded. 

Strongly 

Disagree 

Disagree 

Agree 

Strongly 

Agree 

Don't 

know 

[] 

[] 

[] 

[] 

[] 

9.  To  what  extent  are  process  improvement  goals 
clearly  stated  and  well  understood. 

Not  at  all 

Very 

Little 

Some 

What 

Very 

Much 

Don't 

Know 

[] 

[] 

[] 

[] 

[] 

10.  Regarding  improving  software  development 
processes, 

we  know  what  needs  to  be  improved  but  we  need 

Strongly 

Disagree 

Disagree 

Agree 

Strongly 

Agree 

Don't 

know 

more  guidance  about  how  to  Improve  it. 

[I 

[] 

[] 

[] 

[] 

1 1 .  About  how  much  of  your  work-related  time  did  you  spend 

over  the  past  year  on  process  improvement  related  activities?  _  %  (approximate  percentage) 


12.  Regarding  the  process  improvement 
framework, 

there  are  important  AIS  specific  areas  that  the  CMM 

Strongly 

Disagree 

Disagree 

Agree 

Strongly 

Agree 

Don’t 

know 

does  not  address. 

[] 

[] 

[] 

[] 

[] 

IS.Process  improvement  efforts  detract  from  day  to  day 
operations. 

[] 

[] 

[] 

[] 

[] 

14.  Our  preoccupation  with  software  process  improvement  has 
caused  us  to  neglect  other  important  issues  facing  the 
organization. 

[] 

[] 

[] 

[] 

[} 

15.  Day  to  day  operations  don't  leave  time  for 
software  process  improvement  activities. 

[] 

[] 

t] 

[] 

[] 

16.  Planned  software  process  improvement  initiatives  will 
make  my  job  easier. 

n 

[] 

[] 

[] 

[} 

Questions  17-1 9  relate  to  coordination  and  interaction 
between  functional  areas. 

17.  People  identify  more  with  their  specific  tasks  and 
functions 

than  the  final  product. 

Strongly 

Disagree 

[] 

Disagree 

[] 

Agree 

[] 

Strongly 

Agree 

[] 

Don't 

know 

[] 

18.  Everyone  knows  how  their  work  will  affect  the  work  of  the 
next  person  or  the  quality  of  the  final  product. 

[] 

[] 

[] 

[] 

[] 

1 9.  Boundaries  between  sections  or  departments  interfere 
with  solving  joint  problems. 

n 

[] 

[] 

[] 

[] 

106 


20.  To  what  extent  do  the  following  statements 
accurately 

reflect  the  primary  reason  for  the  software  process 
improvement  effort  at  FSA: 

In  the  future  a  CMM  level  3  certification  will  be 
required  for  DoD  software  developers. 

The  requirement  to  initiate  a  software  process  improvement 
came  from  higher  headquarters. 

Process  Improvement  efforts  will  result  in  FSA  producing 
higher  quality  products  In  a  more  timely  fashion. 

21 .  To  what  extent  Is  each  of  the  following  likely  to  cause  delay 
or  failure  of  the  current  software  process  improvement 

effort  at  FSA? 

Process  improvement  goals  are  not  Integrated 
into  currently  established  FSA  goals. 

Lack  of  customer  support. 

Time  and  resource  constraints.  ^ 

Problems  matching  CMM  requirements  to 
AIS  specific  activities. 

Insufficient  developer  training  and  expertise. 

Overcome  by  other  priorities. 

22.  Which  statement  best  describes  your  attitude 
towards  change? 

Tm  not  sure  howto  change;  besides  things  are  going 
pretty  well  -  maybe  we  don’t  need  to' change." 

"Everybody  gets  frustrated  by  change;  but  you  jjust 
have  to  ignore  the  frustrations  and  push  ahead 
or  you  wont  get  anything  done." 

"You  need  to  change;  you  have  to  change;  in  fact, 

I’d  get  bored  if  we  weren’t  trying  out  new  things." 

"We  change  so  much  around  here  that  we  never 
really  master  things." 


23.  Grade  the  organization  (A,  B,  C,  or  D)  in  the  following  areas: 


Customer  Satisfaction 
Product  Quality 
Job  Satisfaction 


APPENDIX  B.  FSO  SPI  Program 
Post  Level  2  Implementation  Review 
December  1997 


Respondents: 

FSACL 

FSACO 

FSADE 

FSAIN 

FSAKC 

FSAPE 

Management: 

2 

3 

11 

12 

7 

4 

SEPG  Members: 

4 

2 

6 

2 

4 

4 

AIS  Practitioners: 

9 

7 

48 

30 

28 

8 

Support  Division: 

2 

2 

6 

9 

7 

3 

Other: 

13 

4 

0 

Total: 

17 

14 

71 

66 

50 

19 

The  Questions: 


1.  Please  rate  the  senior  management  support  at  the  FSO  for  SPI  (FSO  senior 
management  includes  FSO  Director,  Deputy  Director,  FSO-HQ)  (average  score  on  a  scale 
of  1  to  5): 


FSA/DSE 

Total 

Manaoers 

SEPG 

Members 

AIS 

Practitioners 

Support 

Division 

Other 

FSACL 

3.07 

3.67 

2.75 

3.40 

3.00 

1  DK/NA 

FSACO  • 

3.75 

3.33 

4 

3.85 

2  DK/NA 

- 

FSADE/ 

2.96 

2.18 

2.33 

3.32 

2.67 

DSE-MP 

5  DK/NA 

FSAIN 

3.15 

3.7 

4 

3.24 

2.44 

2.9 

2  DK/NA 

9  DK/NA 

3  DK/NA 

FSAKC 

3.63 

4.00 

3.5 

3.21 

3.33 

2.67 

1  DK/NA 

4  DK/NA 

1  DK/NA 

1  DK/NA 

FSAPE 

3.53 

3.0 

4.5 

3.5 

3.0 

2  DK/NA 

Comments: 

•  Senior  management  is  providing  impetus  and  recognition  for  SPI. 

•  Although  the  rating  is  desired,  CMM  principles  are  not  supported  in  decisions  made. 

•  The  SPI  Program  could  have  been  quicker  and  smoother  if  the  customer  had  been  brought 
“on  board”  sooner.  The  directors  and  AIS  managers  would  have  had  an  easier  job  working 
with  their  AIS  Program  Manager  counterparts  if  support  and  direction  from  DFAS-HQ  were 
more  visible  earlier  in  the  program.  The  lesson  here  may  be  that  even  though  we  were 
changing  FSO  processes  and  procedures,  they  have  an  impact  on  customer  processes  and 
perceptions  of  our  support.  If  they  are  opposed  to  our  direction,  they  can  make  our  progress 
difficult. 

•  As  an  organization,  many  respondents  believed  we  were  attempting  to  implement  too  many 
“programs”  simultaneously  (i.e.,  SPI,  CMIS,  metrics,  LRS,  FPA,  etc.).  These  programs  were 
competing  for  the  same  resources’  time  -  along  with  developing  and  implementing  software 
deliverables.  Many  people  had  “change  overload.”  This  may  mean  that  we  must  be  aware  of 
the  number  and  timing  of  programsAinitiatives  we  are  implementing:  we  have  limited 
resources  that  must  also  product  customer  deliverables. 

•  As  an  organization,  many  respondents  believed  we  stumbled  along  too  much  in  the 
beginning  of  the  program  and  changed  directions  too  many  times  (SMS  versus  CMM  focus. 
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implementing  SMS  task  level  versus  procedure  level,  etc.).  Maybe  this  is  where  piloting 
initiatives  would  benefit  us. 

•  Could  have  provided  better  control  and  oversight,  funds,  and  training.  A  detailed  plan  would 
have  been  nice.  The  “just  do  if  attitude  hasn’t  gotten  us  very  far  in  the  time  the  program  has 
been  in  progress. 

•  I  believe  that  they  support  the  SPI  project  primarily  because  it  is  initiated  from  the  FSO  HQ 
level.  However,  being  so  far  removed  from  the  actual  day-to-day  work.  I’m  not  sure  they  are 
aware  of  the  impacts  of  the  SPI  efforts. 

•  They  do  not  seem  responsible  for  the  amount  of  time  spent  doing  all  the  administrative  tasks. 
The  actual  work  takes  second  place  to  all  the  reporting  and  auditing. 

•  A  great  deal  of  effort  was  put  into  the  initiative,  unfortunately  it  was  not  well  planned  out  in  the 
very  beginning.  A  heavy  price  was  paid  in  later  efforts  due  to  this.  Also,  customer  and  FSA 
personnel  buy-in  to  the  basic  principles  of  SPI  was  never  achieved. 

•  Comments  from  FSAs  are  ignored  by  FSO.  Experience  of  FSA  SEPG  members  is  not 
utilized  by  the  FSO. 

•  From  my  perspective,  it  seemed  to  be  adequate  although  I  was  not  actively  involved  with  • 
FSO  Senior  Management. 

•  FSO  Senior  Management  support;  I  believe  that  the  FSO  staff  have  consistently 
demonstrated  a  real  commitment  towards  achieving  SPI  objectives.  Mr.  Burke’s  visit  was  an 
indication  as  to  the  significance  in  which  the  FSO  views  SPI  activities. 

•  I  don’t  know  who  FSO  managers  are  or  what  they  do. 

•  Mr.  Burke  has  always  presented  a  reasonable  presence  on  SPI  matters.  Others  in  the  FSO 
hierarchy  became  too  focused  on  achieving  the  grade  rather  than  building  the  necessary 
framework  for  continued  improvement. 

•  The  executors  of  SPI  at  the  FSO  HQ  level  approach  SPI  as  a  top-driven  program  to  be 
imposed  rather  than  harnessing  and  focusing  the  grass  roots  desires  and  ideas  for 
improvement. 

•  I’ve  seen  too  many  examples  of  FSO’s  SPI  office  abandoning  processes,  like  CCBs  for  the 
SMS  or  CMIS,  because  of  time  pressures. 

•  I  haven’t  seen  much  direct  support. 

•  Often,  FSO  HQ  does  not  provide  feedback  to  comments  or  concerns  from  the  FSA. 

•  Lack  of  commitment  based  on  resources  and  funding. 

•  I  haven’t  had  direct  contact,  but  support  appears  to  be  very  good. 

•  No  complaints  here.  We  knew  the  SPI  program  was  backed  up  fully,  all  the  way  to  the  top. 

•  SCEs  would  be  more  believable  if  done  by  external  teams. 

•  Policy  statements,  SMS  support,  workshops  were  beneficial. 

•  At  an  AIS  level,  FSO  support  is  not  evident. 

•  At  the  AIS  level,  support  of  the  FSO  senior  management  is  not  clearly  evident. 

•  Why  not  bring  in  advisors  experienced  in  SPI  who  can  work  with  us  regularly. 

•  Barely  visible  in  the  background. 

•  Not  aware  of  FSO  involvement. 

•  We  see  no  evidence  of  their  involvement. 

•  I  really  do  not  know  how  the  powers  work  to  rate. 

•  I  don’t  see  any  support  from  FSO. 

•  Inconsistent,  overpowering,  and  burdensome  at  times,  then  absent  and  lacking. 

•  Support  by  FSO  has  not  been  visible  to  me  other  tan  a  memo  stating  their  support. 

•  Time  was  allotted  and  classes  were  held  with  good  presentations. 

•  Do  not  know  what  they  have  done. 

•  Not  being  privy  to  the  thoughts  of  FSO  senior  management,  I  cannot  answer  this  question. 

•  Have  not  been  involved  yet  in  any  training. 

•  Unknown. 

•  I  don’t  know  who  these  people  are. 

•  I  had  no  contact  at  this  level. 
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•  Not  sure  if  senior  management  actually  understands  what  the  full  ramifications  (such  as  the 
amount  of  documentation)  actually  are. 

•  Initial  support  appeared  to  be  marginal  -  it  improved  once  realized  that  success  depended  on 
team  effort. 

•  Their  views/support  do  not  reach  my  level. 

•  Changes  in  direction  -  no  clear  vision  -  training  and  funding  haphazard. 

•  FSO  senior  management  is  viewed  as  being  a  proponent  of  SPI  and  understanding  the  need 
for  FSO  activities  and  applications  to  implement  a  SPI  program. 

•  SPI  has  proceeded  with  a  “Report  Card”  mentality.  Little  commitment  to  REAL  process 
improvement  beyond  a  few  assessments. 

•  The  Program  is  supported  from  the  top  down. 

•  FSO  senior  management  provided  continuous  vocal  and  administrative  support  to  the  SPI 
effort. 

•  FSO  management  has  spearheaded  the  SPI  program  from  the  beginning,  providing  initial 
funding  for  improvement  initiatives/training. 

•  Not  enough  funding/people  to  correctly  implement  at  FSA  level. 


2.  Please  rate  the  senior  management  support  at  your  FSA  for  SPI  (Site  senior  manager 
inciudes  the  Director  and  Deputy  Director)  (average  score  on  a  scale  of  1  to  5): 


FSA/DSE 

Total 

Manaaers 

SEPG 

Members 

AIS 

Practitioners 

Support 

Division 

Other 

FSACL 

3.69 

4. 

3.25 

3.8 

3.75 

FSACO 

3.86 

3.67 

4 

3.86 

4 

- 

FSADE/ 

3.58 

4.0 

2.83 

3.57 

3.67 

DSE-MP 

FSAIN 

3.35 

3.58 

4 

1  DK/NA 
3.52 

2.67 

3.16 

FSAKC 

3.63 

3.86 

4.00 

5  DK/NA 

3.73 

3.29 

1  DK/NA 
2.67 

FSAPE 

4.06 

4.25 

3.75 

6  DK/NA 
4.17 

4.00 

1  DK/NA 

Comments: 

•  Program  is  supported  by  local  management 

•  Need  more  active  sponsorship  earlier  and  more  frequently  by  the  Division  Chiefs  and 
Directors  (i.e.,  regularly  scheduled  status  updates  from  all  managers  involved  with  SPl, 
requiring  action  and  results,  etc.  -  in  addition  to  the  Director’s  all-hands  meetings).  The 
lesson  here  may  be  that  Division  Chiefs  and  Directors  also  need  education  on  implementing 
change  in  an  organization. 

•  Seems  like  we  are  still  looking  for  ways  not  to  do  things  rather  than  better  ways  to  do  things. 

•  Once  again,  I  believe  that  senior  management  supports  the  SPI  effort  because  of  where  it 
originated.  However,  at  the  local  level,  we  could  use  more  support  in  the  area  of  resources  to 
perform  plus  support  the  procedures  and  process. 

•  Senior  management  participated  in  SPI  as  requested  by  FSA  personnel. 

•  FSA  management  keeps  SPI  an  elevated  concerri. 

•  They  need  to  remain  proactive  instead  of  reactive.  It  would  help  provide  a  clearer 
understanding  and  buy-in  for  the  entire  organization  and  hopefully  less  resistance. 

•  I  believe  that  FSA  Senior  Management  has  consistently  placed  SPI  activities  at  the  top  of  the 
FSACL  priority  list.  The  high  priority  given  to  SPI  objectives  helped  to  communicate 
throughout  our  organization  senior  management’s  commitment  towards  achieving  Level  2. 

•  Additional  support  was  made  available  from  the  systems  support  division  for  some  tool 
development. 
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•  While  the  FSA  management  did  not  fully  embrace  improvement  efforts  initially,  they 
consistently  provided  resources  and  openly  supported  the  program. 

•  SPI  became  much  more  important  when  others  looked  like  they  would  achieve  Level  2  first 
and  when  this  became  a  performance  evaluation  item. 

•  Support  was  consistently  high,  often  at  the  expense  of  developmental  efforts. 

•  Consistently  try  to  educate  us  on  what  is  happening  and  issues  we  need  to  learn  about. 

•  Seems  to  support  this  effort  positively. 

•  SCEs  would  be  more  believable  if  done  by  external  teams. 

•  Hesitant  to  pay  for  effort,  and  some  lack  of  direction  -  perhaps  including  quality  in  estimates 
would  improve  this. 

•  Too  much  emphasis  on  Level  2  and  not  enough  on  improving  our  business. 

•  Seems  that  FSA  management  lost  site  of  goal  -  process  improvement.  Goal  is  now  receiving 
Level  2  with  no  concern  given  to  whether  any  improvement  was  realized.  Mostly  lip  service. 

•  Hardly  any  training  -  not  resourced. 

•  Seem  to  want  the  certification  level,  but  not  the  quality. 

•  Seem  to  support  process-driven  activities,  training,  documentation. 

•  Concern  is  with  reaching  Level  2,  not  improving  process. 

•  The  people  from  division  level  down  are  exceptional.  The  higher  FSA  levels  are  sufficient. 

•  I  don’t  see  any  support  from  senior  management. 

•  Supported  efforts  but  occasionally  lacked  answers. 

•  Support  has  not  been  visible,  but  they  have  not  been  a  hindrance  either. 

•  There  was  sincere  desire  to  achieve  Level  2  of  CMM. 

•  Do  not  know  what  they  have  done. 

•  Not  being  privy  to  the  thoughts  of  FSA  senior  management,  I  cannot  answer  this  question. 

•  Have  not  been  involved  yet  in  any  training. 

•  Constant  emphasis  by  senior  management  on  SPI.  • 

•  Support  is  there  only  because  they  were  told  it  has  to  be  there,  not  because  they  believe  in 
the  process. 

•  We  were  date-driven  by  director.  Not  ready  for  evaluation,  but  had  to  have  one  when  he 
said,  not  when  we  were  ready. 

•  Bob  and  Joe  have  very  strong  support  for  SPI,  and  I  feel  this  has  helped  our  organization. 

•  Conflicting  goals  -  crisis  management  -  no  support  of  SPI  personnel  -  no  awards,  rewards, 
acknowledgment  -  no  funding  -  no  training  -  no  people  -  but  it  still  must  be  in  place  (SPI  and 

.  technical  issues). 

•  FSAPE  senior  management  has  taken  an  active  role  with  implementing  a  successful  SPI 
program. 

•  Provided  resources  for  SPI. 

•  Funding  has  been  provided,  but  the  previous  Deputy  Director  provided  minimal  support  to 
SPI  and  SEPG  activities. 

•  Director  supports  SPI,  but  could  participate  more  actively.  Previous  Deputy  Director  primarily 
supported  only  in  the  last  year. 

•  A  couple  of  “booster”  meetings  would  have  been  helpful.  At  least  as  much  as  for  CFC! 

•  We  were  dedicated  to  achieving  Level  2. 


3.  Please  rate  the  information  related  to  improvement  activities  that  was  made  available  to 
you: 


FSA/DSE 

Total 

■  Manaoers 

SEPG 

Members 

AIS 

Practitioners 

Support 

Division 

Other 

FSACL 

3.56 

3.67 

3.75 

3.40 

3.50 

. 

FSACO 

3.79 

4.33 

4 

3.43 

4 

- 

FSAOe 

3.08 

2.73 

2 

3.41 

2.5 
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DSE-MP 

1  DK/NA 

FSAIN 

3.13 

3.75 

3 

3.27 

3  DK/NA 

2.44 

2.75 

1  DK/NA 

FSAKC 

3.61 

3.86 

3.5 

3.72 

3  DK/NA 

3.43 

2.67 

1  DK/NA 

FSAPE 

Comments: 

3.94 

3.75 

4.0 

4.0 

4.0 

•  Handbook  and  branch  meeting  weekly  topics  were  most  helpful. 

•  The  SMS  training  was  of  little  value  and  the  SMS  is  not  user  friendly  and  is  difficult  to  read. 
The  AISs  started  achieving  “real”  progress  when  they  relied  more  internally  to  train 
themselves  (likely  on  their  documented  processes,  procedures,  and  mechanisms).  Our 
lesson  may  be  two-fold: 

1)  The  FSO  standard  processes  might  be  more  readily  understood  with  fewer  words  -  more 
graphics  and  more  templates 

2)  Guidelines  are  needed.  However,  more  enthusiastic  participation  occurs  when  people 
have 

more  control  in  their  direction. 

•  The  information  was  sufficient  but  what  was  really  necessary  was  insight  to  the  information 
that  was  available. 

•  Our  main  information  for  SPI  was  the  SMS.  This  document  was  often  difficult  to  understand 
plus  difficult  to  tailor  to  our  system.  In  one  specific  case,  the  SMS  did  not  address  a 
necessary  CM  CMM  component  in  enough  detail  to  determine  what  was  needed. 

•  Information,  when  provided,  was  good.  However,  communication  of  information  was 
sporadic  and  not  very  timely.  This  was  a  great  source  of  frustration  to  t  hose  who  tried  to 
further  the  SPI  initiative. 

•  The  SEPG  is  extremely  well  run  and  organized.  Information  is  always  disseminated  to  all 
concerned  as  soon  as  it  is  received. 

•  Early  on  we  were  kept  fairly  current  and  then  little  by  little,  I  began  to  slow  until  currently  we 
receive  no  information  at  all. 

•  The  CMM,  SMS  manuals  and  the  vast  array  of  training  materials  helped  to  establish  a  sound 
foundation  for  developing  and  revising  our  software  development  processes. 

•  FSO  HQ  was  inconsistent  in  providing  information.  While  it  improved  over  time,  many 
important  issues  are  still  not  communicated  to  ALL  affected  individuals. 

•  I  feel  well  informed  in  all  areas  of  improvement  activities.  SMD  and  particularly  Pam  Cromley 
did  an  excellent  job  of  getting  the  word  out  and  keeping  everyone  informed. 

•  Information  was  good  but  sometimes  came  to  us  in  a  round  about  way. 

•  SMS  too  big,  bulky  and  dulled  down  way  too  tar  into  the  software  development  procedural 
routine.  When  emphasis  shifted  to  following  the  CMM,  much  more  relevant  and  user  friendly. 

•  SPI  documents  made  available,  plus  sharing  of  information  within  FSA. 

•  The  SMS  is  all  I  am  aware  of. 

•  Not  sure  what  information  was  made  available.  If  not  sure  about  that,  it  means  that 
communication  may  be  breaking  down. 

•  SMS  was  supposed  to  get  us  to  Level  2  -  It  did  not. 

•  Seem  to  be  behind  the  curve,  trying  to  catch  up.  Forcing,  hurrying  the  process  will  not 
improve  it. 

•  ■  Haven’t  seen  any  material  of  this  nature. 

•  Can’t  get  the  materials. 

•  It  is  overwhelming.  Umpteen  versions  of  multiple  documents.  The  tail  wags  the  dog.  I 
cannot  keep  track. 

•  Co-workers  seem  to  be  saddled  with  responsibility  of  supporting/establishing  SPI  efforts  with 
little  or  no  support  from  senior  management. 

•  Generally  tempered  with  pragmatic  solutions. 

•  Certainly  was  plentiful. 
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•  Too  wordy.  Would  be  more  helpful  to  have  visual  materials  to  relate  information. 

•  It  was  presented  in  a  way  that  we  do  our  work. 

•  Each  of  us  given  our  own  copy  of  guide  to  follow. 

•  Have  not  been  involved  yet  in  any  training. 

•  Unknown. 

•  Enough  information  has  been  given  to  have  a  basic  understanding,  but  not  enough  to  do  the 

work  correctly. 

•  SEI  has  it  down  to  a  science. 

•  New  information  hard  to  get  -  need  news  flashes,  bulletin  board,  or  a  formal  distribution 

system  starting  with  FSO. 

•  The  SEI  information  proved  quite  helpful. 

•  FSO  provided  excellent  information  through  UPDATE,  SPICON  and  Email. 

•  Interaction  with  other  FSAs  and  the  FSO  has  been  invaluable  to  the  SEPG. 

•  Normal  workload  doesn’t  allow  us  to  keep  up  with  this  information  on  a  regular  basis. 

•  Meetings  were  held  and  reading  material  made  available. 

•  Report  was  very  good. 

4.  The  most  significant  improvement  caused  by  SPI  efforts  was: 

•  estimating  process 

•  use  of  repeatable  processes  for  SCM  and  project  management 

•  documented,  accessible,  well-understood  processes  and  procedures  (improvements 
in  approach  center  more  on  the  CMM  versus  SMS) 

•  reviews  helped  identify  questions  and  issues  earlier 

•  tracking  has  improved 

•  better  support  for  firm  fixed  prices  and  schedules 

•  better  control 

•  the  set  up  of  the  CM'-  with  so  many  pieces  and  parts  involved  in  any  given  release,  it 
is  really  nice  to  have  a  central  area  of  control 

•  Better  tracking  and  oversight 

•  metrics,  better  collection  of  data  for  oversight  and  reporting 

•  institutionalizing  the  practice  of  having  programmers  document  the  location  of  their 
changes  plus  putting  those  changes  on  a  release  PDS.  If  any  programmer  is  not 
available  on  implementation  day,  any  other  programmer  can  locate  the  program 
changes  plus  implement  them 

•  the  stabilizing  and  reliability  obtained  in  the  customer  testing  region.  We  now  are 
aware  of  all  changes  being  operated  on,  all  changes  being  tested  as  a  package. 
Also,'baselining  of  requirements  and  software  was  beneficial. 

•  enhancement  of  project  management  activities. 

•  system  releases  and  bounding  of  requirements. 

•  getting  a  large  number  of  employees  to  at  least  learn  about  SPI  and  getting  them  to 
accept  and  work  in  a  release  mode. 

•  management  has  an  effective  way  to  show  all  work  effort  related  to  a  particular 
release. 

•  a  sense  of  organization  about  what  is  in  a  release  and  when  it  will  be  implemented. 
This  begins  with  early  planning  and  builds  from  there-. 

•  better  control  with  configuration  management.  Clearer  picture  of  the  status  of 
projects  with  PM  tracking  and  oversight  process 

•  I  think  it  organized  procedures  and  provided  a  clear  and  concise  way  of  doing 
business  along  with  the  establishment  of  data  archive  and  feedback  thus  providing 
continued  support  for  future  projects 

•  project  planning  activities  are  better  defined  and  communicated: 

•  software  development  activities  have  become  more  uniform 

•  software  requirements 

•  system  components  (CIs)  are  more  accurately  defined  and  documented 
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•  the  project  release  cycle  allows  more  time  for  requirements,  programming 
and  testing  to  all  do  a  better  job 

•  implementation  activities  have  been  improved  and  better  documented 

•  communication  has  been  improved  through  the  FSA  as  well  as  with  our 
customers 

•  It  has  enhanced  our  coverage  for  implementing  changes  when  people  are  out  by 
creating  a  more  formal  procedure,  although  we  were  addressing  that  as  a  risk  before. 

•  documentation  of  processes  and  centralizing  of  paperwork 

•  A  unification  of  effort  within  the  targeted  system.  SPI  helped  break  down  some 
boundaries  while  addressing  a  common  issue.  The  same  type  of  coming  together 
has  begun  at  the  FSO  level. 

•  Institution  of  standard,  formal  estimating  methods,  and  a  more  formal  process  for 
negotiating,  scheduling,  and  tracking  system  releases. 

•  Bringing  structure  into  a  chaotic  environment.  Documenting  and  producing 
processes  was  greatly  needed. 

•  Attaining  Level  2. 

•  Single  point  of  entry. 

•  In  the  requirement  management  area. 

•  PANAPT 

•  Getting  people  to  think  outside  “the  box”  of  their  world  and  focus  their  collective 
energies  in  a  united  front  to  increase  the  quality  of  software  developed  and 
maintained  for  their  customers.  Discussion  and  action  to  improve  processes. 

•  Passing  information  to  practitioners  needed  to  achieve  Level  2. 

•  More  control  over  projects,  errors  caught  earlier  in  the  process. 

•  Processes  in  place  that  force  us  to  pay  more  attention  to  the  products  earlier  in  the 
lifecycle. 

•  A  heightened  awareness  of  the  flow  of  work  and  more  accurate  measuring  of  SCRs. 

•  Repeatability!  Manageability!  Consistency! 

•  Openness  to  process  and  improvement  and  therefore  change  -  improved 
communication  resulted  in  quality  products. 

•  Developing  standard  procedures  for  AIS 

•  Communication  and  coordination 

•  Establishment  of  a  goal/direction  to  work  toward  -  through  group  efforts,  definition  of 
how  we  would  reach  goals 

•  Communication  outside  division  boundaries  has  increased 

•  Documented  procedures 

•  Getting  CMM  process  underway  with  several  AISs  reaching  the  Level  2  plateau 

•  People  organized  in  their  jobs  -  they  are  more  accountable  -  SPI  requires  everything 
to  be  documented  and  followed 

•  Awareness,  assessment,  training 

•  Don’t  know 

•  The  knowledge  of  the  SEI’s  CMM  process  improvement 

•  Getting  the  requirements  from  the  customer  in  writing 

•  Systematic  development/maintenance  efforts  -  repetitive 

•  Repeatable  processes  used  by  everyone 

•  Reviews  seem  to  catch  errors  earlier 

•  We  will  meet  Level  2  requirements 

•  Getting  clearer  requirements  for  change  requests  -  improving  understanding  between 
user  and  support  groups 

•  Documenting  many  of  the  procedures  already  in  place  -  providing  more  uniform 
adherence 

•  Standardized  and  repetitive  processes  have  increased  productivity  -  better 
understanding  among  branches,  functionals,  and  technicals  -  improved 
communication  between  FSA  centers. 
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•  Producing  a  better  product  the  first  time 

•  Crucial  procedures  needed  to  make  changes  are  thoroughly  defined/documented  - 
work  more  orderly  and  repeatable 

•  Quality 

•  More  knowledge  about  what  affects  a  project’s  progress  -  increased  communications 
among  technical,  environmental,  management,  etc.,  members 

•  Improved  our  awareness  that  most  of  what  we  do  is  already  Level  2  and  repeatable  - 
enhanced  quality 

•  More  documentation  and  tracking  of  software  changes 

•  Documentation  on  the  project 

•  Standardization 

•  Gets  entire  team  involved  in  development  efforts 

•  Proposed/actual  implementation  of  SMS  and  development  of  CMM  to  bring  every  AIS 
up  to  Level  2  and  beyond 

•  Documenting  on  going  towards  processes  and  procedures 

•  Awareness  of  need  to  document  processes 

•  A  focus  on  documenting  processes  and  then  improving  them 

•  Have  not  been  involved  with  any  SPI  efforts  to  date 

•  Don’t  know 

•  Better  quality 

•  Better  impact  analysis 

•  Good  planning  efforts  which  includes  start  to  end  management  -  good  training  plan  - 
good  communication 

•  Process  definition 

•  Formal  reviews 

•  Management  of  requirements  and  project  management  improved  drastically 

•  On  schedule  -  on  budget 

•  Strengthen  the  team  effort  and  document  our  procedures 

•  Reduction  in  errors/problems  in  the  finished  product 

•  Testing  procedures 

•  Communications 

•  Requirements  and  design  are  more  completely  documented 

•  Getting  business  practices  documented  -  established  standards  for  all  software 
engineers  to  follow  and  a  guide  for  reference 

•  Worked  as  a  team  and  all  are  doing  the  same  way  now 

•  Proactive  instead  of  “fireman”  efforts  for  a  highly  active  production  system 

•  Documentation  of  existing  processes  and  insurance  everyone  understands  and  uses 
the  processes 

•  The  micromanagement  of  processes  has  caused  potential  problems  to  arise  and  be 
fixed  before  they  become  a  major  issue 

•  Uniformity  in  process 

•  Documentation  of  processes  and  projects  -  we  can  teach  others  more  easily,  move 
personnel,  and  review  processes  for  problems  or  improvements 

•  Business  processes  which  had  been  used  for  some  time  were  documented. 
Employees  were  made  aware  of  current  approaches  for  evaluating  system 
development  projects. 

•  Discipline  in  developing  software,  scheduling,  sizing. 

•  More  working-level  people  were  made  aware  of  need  for  process  improvement. 

•  An  awareness  of  process  improvement  and  I  truly  believe  a  desire  by  practitioners  to 
improve. 

•  Two  major  AISs  achieved  Level  2  on  first  evaluation. 

•  More  process  awareness,  more  team  orientation,  better  communication. 

•  The  roles  and  responsibilities  of  other  SEPG  members  are  known  and  understood. 
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•  Development  of  training  plans. 

•  Documentation  of  the  DCPS  process. 

•  More  individual  involvement  in  the  total  process. 

•  The  current  release  of  the  software  is  a  much  better  and  cleaner  product  than  would 
have  been  possible  without  SPI. 

•  Opened  communication  between  different  levels. 

•  Consistent  methods  for  handling  both  routine  and  emergency  SCRs. 

•  Better  Quality  Assurance  and  development  of  a  software  process  handbook  for  easy 
access  to  materials/procedures  used  throughout  the  project. 

•  People  are  more  process  oriented. 

•  Better  communications  on  the  project,  better  understanding  of  goals  and  objectives 
and  better  quality  product. 

•  Regimented  software  engineering  practices  implemented  -  a  standard,  organized, 
measured  method  of  work. 


5.  The  most  significant  negative  impact  caused  by  SPi  efforts  was: 

•  increased  release  time  and  ability  to  make  changes  quickly 

•  emphasis  placed  on  process  at  the  expense  of  “nuts  and  bolts”  of 
producing  software 

•  too  much  paperwork 

•  need  more  customer  commitment  and  support  of  SPI.  The  program  is  too  costly  and 
important  to  FSO  not  to  gain  strong,  active,  early  commitment  starting  at  the  top  of 
the  DFAS  management  chain. 

•  The  various  support  tools  were  not  very-user-friendly  (CMIS,  SMS  Survey  Tool, 

LRS).  We  needed  more  top-down  customer  management  support  of  CMIS. . 

•  Too  much  additional  (and  sometimes  duplicated)  paperwork  and  meetings  (preparing 
for  and  developing  status  reports  to  be  reviewed  by  multiple  levels  of  management  - 
from  branch  chief  up  to  FSO-HQ  director). 

•  Lines  of  code  projections  - 1  really  don’t  know  how  our  projects  were  estimated  prior 
to  SPI,  but  we  appeared  to  get  things  done  within  budget. 

•  Increase  in  costs 

•  The  perception  that  not  achieving  the  next  level  after  the  first  audit  is  a  failure.  A  plan 
that  did  not  emphasize  success  or  failure  but  long-term  improvement  would  have 
avoided  a  lot  of  negativity  and  bad  feelings  that  resulted  from  the  assessment. 

•  The  overhead.  A  tremendous  amount  of  overhead  has  been  added  to  our  day-tO:day 
business  operation.  At  times  the  process  overshadows  plus  inhibits  the 
implementation  of  a  quality  product. 

•  There  is  too  much  administrative  overhead,  i.e.,  too  many  forms  to  fill  out,  too  many 
checklists,  etc.  All  these  processes  have  not  produced  a  better  quality  product. 

•  Staff  morale 

•  Sense  of  frustration  due  to  lack  of  buy-in  to  the  basic  principles  of  SPI.  This  affected 
proponents  and  detractors  of  SPI  alike. 

•  Cost  too  high,  cost  to  implement  and  maintain 

•  The  amount  of  time  spent  on  determining  how  to  implement  the  CMM,  even  though 
we  started  with  the  SMS 

•  COST,  COST,  COST!  FSO  has  an  unreasonable  view  on  funding  for 
implementation,  especially  for  small  systems. 

•  There  is  a  huge  “overhead”  and  much  of  this  has  minimal  pay-back.  There  are  many 
things  we  are  doing  just  because  they  are  necessary  to  be  Level  2. 

•  Increase  in  cost  for  SPI  efforts 

•  The  general  feeling  is  of  an  overwhelming  additional  workload  from  the  required 
record-keeping  and  tracking  of  projects.  Many  feel  it  is  unproductive  and  taking  away 
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time  that  should  actually  be  spent  programming  and  is  in  fact  jeopardizing  not  only 
the  project,  but  the  entire  organization. 

•  The  negative  side  is  that  software  modifications  require  a  greater  amount  of  time  to 
complete. 

•  SPI  has  added  a  great  deal  of  overhead  with  little  value  added 

•  Paperwork  gets  in  the  way  of  the  actual  projects. 

•  Time-consuming  up  front. 

•  Resulted  in  duplication  of  effort,  entering  similar  information  into  several  systems. 

•  A  lot  of  effort  was  spent  formalizing  processes  without  improving  them.  Addressed 
the  “book”  solution  rather  than  the  needs  of  the  AIS.  Changes  were  imposed  rather 
than  drawn  from  the  AIS.  There’s  a  lot  of  resistance  at  the  practitioner  level  in  those 
areas  where  they  do  additional  work  without  apparent  benefit. 

•  With  the  pressure  on  to  make  SPI-related  deadlines,  competition  for  resources  within 
the  FSA  has  been  bitter.  While  I  have  heard  some  very  positive  comments  from 
some  developers,  many  still  see  SPI  as  a  drain  on  their  dwindling  time. 

•  I  did  not  detect  any  negative  impact  caused  by  SPI  efforts.  Every  effort  made  toward 
improvement  is  a  positive  effort. 

•  That  it  was  implemented  nearly  concurrently  with  fee-for-service  and  was  used  as  a 
“standardization  mechanism.” 

•  Great  time  involved  doing  process  meetings  when  a  mission  is  time-critical.  It 
appears  at  times  we  could  be  better  served  just  doing  our  job  rather  than  attending 
meetings. 

•  Initial  resistance,  however,  it  seems  most  employees  can  see  a  benefit  now. 

•  Stress  due  to  unclear  requirements. 

•  Manpower  decreased  when  specialists  were  lost  to  create/coordinate  SPI  and 
this  has  increased  the  workload  on  the  remaining  heroes. 

•  There  must  be  a  commitment  to  the  time  involved  in  order  to  reap  the  quality 
benefits. 

•  Confusion  caused  by  lack  of  good  examples  of  what  documents  and  procedures 
should  be  developed. 

•  Money  and  time  - 1  have  not  seen  any  return  on  the  SPI  investment  yet 

•  Lack  of  available  training  to  be  more  efficient  in  our  efforts 

•  Monetary  investment  has  shown  minimal  improvement  -  significant  number  of  high- 
salaried  employees  devoted  to  SPI.  Significant  overhead  has  not  shown  ROI  -  too 
much  $$,  too  little  improvement 

•  SEPG  that  existed  early  on  was  not  a  help  -  their  attitude  negatively  impacted  SPI 

•  Impression  that  projects  served  FSA  staff  and  through  them  the  FSO  instead  of  the 
other  way  around 

•  FSO  preaches  SPI  -  no  support  given  to  local  SPI  efforts 

•  Amount  of  time  spent  on  activities,  juggling  priorities  (SPI  versus  customer 
satisfaction) 

•  Don’t  know 

•  Impact  on  morale  caused  by  the  half-hearted  implementation 

•  Generated  many  forms,  steps  and  procedures  which  add  a  lot  of  time  to  completing 
the  work. 

•  Adjustments  of  overhead 

•  Functional  community  not  receptive  to  SPI 

•  Customer  complaints 

•  T remendous  overhead  costs  -  affects  productivity 

•  A  proliferation  of  paperwork  and  requirements  without  any  clear  goals  or  direction 

•  Time  involved  to  make  even  small  changes 

•  Time  spent  seemed  excessive  -  archival  of  documentation  is  excessive  -  find  another 
way  of  providing  evidence 
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•  Making  changes  to  processes  after  an  AIS  placed  in  production  caused  some 
confusion  &  duplicate  efforts 

•  Morale  problems  -  insufficient  information  received  at  lower  levels  -  people  think  this 
involves  major  changes  in  how  they  work,  but  this  isn’t  true  for  everyone 

•  T akes  longer  to  get  the  job  done 

•  Sometimes,  especially  for  simple  changes,  SMS  slows  the  completion  of  work  tasks 
too  much 

•  Increased  amount  of  time  it  takes 

•  Requested  changes  can  no  longer  be  accomplished  quickly  -  spend  much  more  time 
documenting  and  waiting  for  other  people  to  finish  their  steps 

•  Increased  time  allotted  for  release 

•  It  takes  much  longer  to  make  changes  in  software  and  minor  improvements  are  not 
made 

•  Added  paperwork  that  needs  to  be  charged  to  a  customer  who  already  doesn’t  have 
enough  money  to  get  job  done 

•  Significant  amount  of  time  expended  in  reviews  and  documentation 

•  Amount  of  upfront  time  expended  to  implement  SMS  and  CMM 

•  No  interaction  to  determine  what  the  processes  are  and  what  doesn’t  work  or  needs 
improvement 

•  Confusion  as  to  what  projects  to  apply  SPI/SMS  standards 

•  Documentation  was  distributed  and  expected  to  be  followed,  but  no  formal  training 
was  held 

•  Extra  paperwork 

•  Many  SCEs  have  required  additional  funding  to  complete  SPI  activities 

•  Lack  of  “proactive”  support  from  senior  management 

•  Morale 

•  Theamountof  documentation  required  to  satisfy  the  requirements 

•  Expensive  to  implement  -  costly  record  keeping 

•  Customer  has  claimed  slower  turnarounds  for  impacts  and  releases 

•  The  layers  of  extra  work  and  bureaucratic  red  tape  that  are  necessary  to  meet  SPI 
requirements  often  seem  to  be  a  make-work  waste  of  time,  especially  for  small 
changes/SCRs  . 

•  The  reduction  on  production  because  of  the  time  spent  on  doing  documentation  in 
support  of  SPI  efforts  -  SPI  requires  2  or  3  times  the  amount  of  time  spent  on 
documenting  each  step  or  phase  -  more  attention  on  meeting  SPI  level  than  on  our 
job 

•  Terrible  cost  -  too  much  documentation 

•  Time  required 

•  Lateness  in  getting  procedures  on  how  to  do  tasks  -  with  no  initial  direction,  much 
was  done  without  guidance 

•  Time/$$  to  develop  all  processes/procedures  -  could  have  used  a  Level  2  AIS’s  stuff 
and  gotten  there  quicker  with  less  $$  -  share  stuff  -  director  needs  to  realize  SPI 
takes  time  and  shouldn’t  be  rushed  or  date-driven 

•  Customer/functional  staff  not  willing  to  get  it  done  “now”  -  moving  train  and  getting  it 
painted  syndrome 

•  Effort  to  follow  SMS  as  the  methodology  for  reaching  CMM  Level  2 

•  Cost  and  time 

•  Lip  service  by  senior  management  -  suspension  of  staff  functions  indicates  non¬ 
support  of  SPI  effort  -  no  management  support  -  too  many  lies  -  our  heroes  are 
dropping  like  flies  -  they  just  give  up 

•  Overburden  on  AIS  staff  -  unrealistic  or  uncompromising  dates  and  goals  were  set 
without  input  from  the  AIS  -  we  are  understaffed  in  some  areas,  and  this  makes  it 
worse. 
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•  Some  “game-playing”  on  the  part  of  managers  who  focus  on  appearance  of 
improvement  rather  than  real  improvement. 

•  Additional  time  to  develop  various  plans  and  products. 

•  The  initial  impact  was  the  vast  quantity  of  paper  generated,  however,  as  we  progress, 
that  should  level  out. 

•  Initial  perception  was  of  “program”  to  be  forced  on  projects. 

•  Feeling  that  the  reviews  and  comments  associated  have  negatively  affected  the 
teamwork  attitude  the  system  developers  need. 

•  Time  required  to  fill  out  forms,  develop/maintain  documentation  reduces  the  amount 
of  work  that  can  get  done  from  one  update  release  to  another  even  though  the  “old” 
way  might  have  resulted  in  a  10%  rework  and  re-release. 

•  Delay  of  a  Release  was  caused  by  the  preparation  of  the  SCE. 

•  The  additional  meetings  and  papenwork  are  very  time-consuming  and  lengthens  the 
time  to  complete  work. 

•  Increased  response  time  for  handling  emergencies  caused  by  need  to  follow 
procedure. 

•  Upkeep  of  the  Business  Handbook  is  added  workload. 

•  Good  initiatives  start  out  but  without  proper  funding/value  are  quickly  overcome  by 
events. 

•  More  paperwork. 

•  Very  time-consuming  -  cost/benefit  still  undetermined.  Customers  leery. 


6.  Please  rate  the  training  you  received  on  SPi  Initiatives: 


FSA/DSE 

Total 

Manaaers 

SEPG 

Members 

AIS 

Practitioners 

Support 

Division 

Other 

FSACL 

3.56 

3.67 

3.5 

3.6 

3.5 

FSACO 

3.43 

3.67 

4 

3 

4 

- 

FSADE/ 

3.00 

3.18 

1.6 

3.13 

2.83 

DSE-MP 

FSAIN 

3.14 

3.27. 

2 

1  DK/NA 

3.38 

2.67 

2.9 

FSAKC 

3.54 

1  DK/NA 
3.57 

1  DK/NA 
3.5 

1  DK/NA 

3.68 

3.57 

3  DK/NA 
2.33 

FSAPE 

3.75 

3.75 

3.75 

3  DK/NA 

3.50 

4.50 

1  DK/NA 

1  DK/NA 

Comments: 

•  SPI  training  was  at  appropriate  time/level 

•  More  specific  training  is  needed  (how  the  AIS  will  specifically  perform  the 
processes/procedures  versus  the  what  in  the  SMS;  how  to  implement  the  SMS/CMM 
(perhaps  from  the  SEPG);  how  to  translate  the  CMM/SMS  into  tangible  mechanisms 
(checklists,  forms,  etc.) 

•  “Shadow  consulting,"  change  agent  training  and  more  descriptions  of  specific 
benefits  for  each  KPA  versus  general  benefits  of  SPI  were  needed. 

•  Customer  at  the  site  level  needed  “education”  on  SPI  benefits  -  both  generally  and  by 
KPA. 

•  Senior  managers  (division  chiefs  and  directors)  could  have  benefited  from  training 
related  to  implementing  change  in  organizations 

•  Received  a  lot  of  training  that  gave  a  lot  of  information,  but  very  little  insight.  Could 
have  been  better. 

•  Training  was  received  on  each  KPA.  This  training  was  provided  by  FSO.  I 
personally  received  some  excellent  training  from  an  outside  vendor,  FASTRAK  for 
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CM.  Perhaps  some  sharing  among  AISs  or  visits  to  see  the  process  in  action  would 
have  been  helpful.  There  has  been  no  training  since  approximately  2  years  ago. 

•  Training  was  received  3  years  ago.  Nothing  since. 

•  Training  provided  was  okay.  The  problem  was  that  due  to  inadequate  up-front 
planning,  the  training  needed  by  some  was  not  provided  until  much  too  late,  e.g., 
TWG  members  working  on  process  definition. 

•  T raining  received  varies  greatly  and  not  necessarily  at  the  working  level. 

•  Outside  vendor  training  has  been  excellent. 

•  Overall  in-house  coupled  with  vendor  training  proved  very  beneficial. 

•  The  training  received  was  very  complete.  It  was  extremely  helpful  in  providing  a 
thorough  comprehension  of  what  was  required  for  each  KPA. 

•  T raining  provided  by  SEI  and  other  vendors  was  extremely  beneficial.  T raining 
provided  by  the  FSO  was  weak, 

•  Some  goo^some  bad  to  average  in  the  middle.  For  the  most  part,  internally 
developed  training  was  gbod/FSO-developed  training  was  bad. 

•  Working  primarily  on  Requirements  Management  -  requested  but  could  never  get 
formal  training  on  this  KPA  -  feel  it  caused  a  lot  of  wasted  time. 

•  initial  overview  training  was  dull,  dry,  boring.  Didn’t  like  being  read  to  from  a  book 
that  I  had  in  front  of  me.  Learned  more  by  being  involved  with  a  local  working  group 
and  the  later  training  sessions  given  by  the  Change  Agents  and  other  groups. 

•  It  took  several  sessions  to  get  it  all  straight  in  mind,  but  I  think  I  finally  got  there. 

•  Explaining  evaluation  system  helped  tremendously. 

•  Need  more  whys  and  theory  versus  reading  CMM/SMS  -  Gini  provided  good  review 
of  SMS/CMM 

•  STC  good  as  a  starting  point,  but  flat  overall  -  could  have  been  more  insightful  and 
less  text  -  vendor  training  veiy  good  -  should  have  KPA  experts  within  FSA  facilitate 
training 

•  Vague  information  -  sessions  would  usually  degenerate  into  complaining  about  the 
proponent  agencies 

•  Very  little  training  was  available  -  no  money  in  budget 

•  Several  requests  made  for  vendor  training  in  CM  and  SQA,  but  all  denied  due  to  lack 
of  funds. 

•  Had  to  obtain  own  -  STC  training  very  poor 

•  T raining  from  Learning  T ree  emphasized  testing  -  training  from  FSAIN  staff  was 
simply  a  reiteration  of  CMM/SMS  -  my  time  better  spent  in  reading  primary  source  - 
training  from  STC  (ENSCO)  was  very  good 

•  CMIS  class  done  well,  however,  without  specific  examples  of  using  it  in  real  life,  I 
don’t  see  what  to  do  with  it 

•  Still  waiting 

•  Never  received  training 

•  I  have  not  received  any  training  for  SPI 

•  I  have  been  trained  elsewhere,  no  training  here 

•  When  you  don’t  get  the  material,  you  cannot  learn  as  much 

•  We  have  some  very  talented  people  in  our  organization 

•  Am  not  asking  for  more  - 1  learn  by  putting  down  on  paper  in  words  I  understand, 
clarifying  and  freedom  to  study  -  rny  learning  is  slow  but  sure  -  it  is  not  good  for  in- 
house  training 

•  All  theoretical,  not  practical  -  learned  in  class  -  not  used  in  work  place  =  soon 
forgotten 

•  Occasionally  too  detailed  and  long 

•  Continuous  training  or  awareness  sessions  are  very  important  and  should  be  done 
more  frequently 

•  Training  received  at  various  times,  but  nothing  implemented  yet 

•  I  like  the  SPI  demo  on  software 
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•  Much  redundancy  bordering  on  overkill,  but  the  SMS  process  was  well  explained 

•  These  topics  need  to  be  scrutinized  to  make  more  understandable 

•  Many  handouts  were  given,  and  several  classes  covered  what  was  needed  to 
achieve  Level  2 

•  After  given  the  training,  don’t  use  for  months,  if  ever 

•  Don’t  think  I  received  any  formal  training,  and  any  informal  training  was  not  tagged  as 
SPI 

•  Could  not  attend  class 

•  Need  training  for  all  contract  personnel 

•  Need  specific  training  in  the  assigned  AIS 

•  Needed  more  CMM  training  up  front 

•  Extremely  dry  and  boring  usually 

•  Unable  to  rate  for  no  basis  for  evaluation 

•  Need  more 

•  Training  given  and  then  expected  to  get  it  done  right  away  -  How  do  you  roll  over 
your  environment  when  you  are  always  in  a  50  hour  work  week  in  40  hours  before 
SPI 

•  Training  I  received  was  totally  inadequate  -  training  needs  to  focus  on  system  specific 
rather  than  organizationally 

•  STC  staff  has  been  dedicated  and  committed  to  delivering  quality  training  -  tough 
mission 

•  Why  cancel  the  STC  training  group? 

•  While  my  training  was  good,  we  are  having  difficulty  getting  training  for  CM  and  QA 
personnel  as  well  as  other  SPI  training  for  SCE 

•  The  training  provided  a  good  understanding  of  the  SPI  program  and  evaluation 
process. 

•  Some  areas:  sizing,  tracking,  oversight,  seemed  shallow. 

•  Received  valuable  extra  training  from  SEI  due  to  SEPG  and  SCE  evaluator  role. 

•  FSO  training  was  sufficient,  but  SPI  conferences  were  exceptional. 

•  Training  was  ample  and  conducted  by  personnel  very  knowledgeable  of  SPI. 

•  Have  only  received  in-house  training. 

•  Limited  access  to  within  Activity  Level  2  systems  documentation  for  guidance  and  to 
knowledgeable  personnel  for  documentation  development.  We  had  to  start  from 
scratch  even  though  our  Activity  had  a  Level  2  System. 

•  We  only  had  it  before  the  CMM  audit. 

•  I  feel  that  the  training  I  have  received  on  SPI  has  been  beneficial.  We  also  have  a 
SPI  bulletin  board.  Local  training  was  very  good. 


7.  Please  rate  the  SEI  support  you  received  on  SPI  Initiatives: 


FSA/DSE 

Total 

Manaoers 

SEPG 

Members 

AIS 

Practitioners 

Support 

Division 

Other 

FSACL 

3.64 

3.67 

4 

3.25 

1  DK/NA 

3.67 

1  DK/NA 

FSACO 

3.46 

4 

4 

3.4 

2  DK/NA 

4 

1  DK/NA 

■ 

FSADE/ 

DSE-MP 

2.91 

3 

2.5 

3.03 

8  DK/NA 

2.4 

FSAiN 

3.05 

3.67 

3DK/NA 

3. 

1  DK/NA 

3.06 

8  DK/NA 

2.83 

3  DK/NA 

2.56 

4  DK/NA 

FSAKC 

3.45 

3.17 

1  DK/NA 

3.25 

3.95 

6  DK/NA 

2.57 

2.67 

1  DK/NA 
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FSAPE 


3.67 


4.0 


4.0 


3  67  3  2 

1  DK/NA  1  DK/NA 


Comments: 

•  SEI  was  very  helpful 

•  Not  much  contact  by  AIS  personnel  with  SEI  -  some  by  ccmail 

•  Support  at  the  SCEs  was  excellent 

•  More  direct,  hands-on  support  from  SEI  at  the  site  level  (shadow  consulting,  etc.) 
would  have  been  beneficial.  The  lesson  here  may  be  that  the  SEPG  and  site  needs 
more  consultation  support  to  facilitate  quicker  and  easier  SPI  implementation.  This 
doesn’t  have  to  be  full-time,  but  periodic  consultation  at  critical  stages. 

•  Did  not  have  much  interaction  with  the  SEI  personnel.  I  know  that  some  SEI  reps 
came  here  to  help  us  with  reaching  Level  2  in  the  area  of  PM.  However,  I  cannot  rate 
the  support  given. 

•  SEI  has  been  very  cooperative  with  lending  support. 

•  Very  beneficial  when  the  opportunity  arises  that  one-on-one  communication  is 
possible.  At  other  times,  when  an  issue  is  raised  with  hopes  of  comments  from  SEI, 
that  is  a  little  tougher  to  accomplish. 

•  SEI’s  support  was  not  extremely  visible  to  me.  I  would  have  to  say  that  the  majority 
of  our  support  came  either  from  the  FSO  or  FSA  level. 

•  What  we  were,  told  we  had  to  do  to  become  Level  2  in  PM  did  not  mesh  with  what 
other  AISs  were  doing  who  were  Level  2.  Do  not  agree  with  SEI  opinion  expressed 
that  “you  cannot  do  too  much  ...”  -  in  this  case  PM,  but  seems  to  be  a  common 
theme. 

•  The  shadow  consulting  we  received  was  invaluable. 

•  Rozum  was  knowledgeable  and  ready  to  assist  when  needed. 

•  SEI  who?  NO,  actually,  it  was  good  at  times,  but  often  resulted  in  confusion  from  the 
mixed  messages  that  were  received. 

•  Support  has  always  been  very  helpful. 

•  I  have  never  been  clear  on  the  scope  of  the  SEI  agreement.  Would  they  have  helped 
me  on  my  initiatives  if  I  had  asked  them?  I  felt  SEI  was  very  short  on  concrete  advice 
-  they  were  more  like  a  stereo-typical  therapist,  answering  “well,  what  do  you  think 
you  should  do?”  to  every  question.  We’d  have  been  better  off  on  our  own. 

•  My  position  as  practitioner  didn’t  put  me  into  contact  with  SEI. 

•  Don’t  know  if  any  support  received  at  project  level 

•  Available  to  answer  questions  and  analyze  solutions  -  early  on,  wished  more  direct 
answers 

•  None,  other  than  having  a  copy  of  the  CMM 

•  Never  had  direct  contact  with  SEI 

•  Hardly  ever  here  -  never  got  direct  answers 

•  I  have  not  received  any  SEI  support 

•  Not  aware  of  SEI  involvement 

•  I  have  not  received  any  SEI  support 

•  I  believe  some  SEI  support  could  be  very  helpful  to  the  effort 

•  Learned  from  others  in  the  office  and  trial  and  error 

•  Written  materials  only,  no  explanation 

•  I  have  had  no  contact  with  SEI 

•  Co-workers  and  branch  chiefs  are  supporting  effort  without  much  senior  management 
help 

•  Too  academic 

•  Not  aware  of  what  they  have  done 

•  I  personally  know  of  no  SEI  actions  although  they  would  have  taken  place  at  a 
different  level 

•  Questions  were  answered  in  a  timely  manner 
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•  Not  much 

•  Have  not  read  CMM 

•  Unknown 

•  More  SPI  support  would  be  helpful  -  they  should  monitor  our  practices  more  closely 
and  give  us  guidance  on  what  improvements  need  to  be  made 

•  They  were  afraid  (so  it  appeared)  to  give  yes  or  no  answers  when  it  came  to 
procedural  questions 

•  What  does  SEI  stand  for? 

•  I  have  no  contact  at  this  level 

•  Much  has  been  left  to  the  users  of  the  procedures  to  guess  at 

•  SEI  members  were  learning  processes  and  were  inflexible  in  implementation  of  the 
SMS 

•  They’ve  been  at  our  beck  and  call  and  have  always  responded/supported  us 

•  Too  many  flip-flops  on  how  we  do  our  business 

•  We  haven’t  had  any  yet  on  our  project 

•  SEI  support  helped  to  clarify  terminology  and  facilitate  the  crosswalk  from  current 
practices  to  the  CMM. 

•  SEI  provided  continuous  on-site  and  off-site  support  that  was  a  significant  factor  in 
improving  software  engineering  practices  to  Level  2. 

•  They  were  very  helpful  in  zeroing  in  on  what  needed  to  be  done  for  an  assessment. 

•  SEI  sent  a  representative  to  do  an  evaluation  prior  to  the  SCE.  His  input  proved  to 
be  critical  in  many  areas. 

•  We  had  one  sit-down  session  with  an  outside  SEI  person  in  the  8-12  month  process. 


8.  Please  rate  the  FSO  support  activities  to  SPI  Initiatives  (These  activities  include 
training,  FSO  SPI  conferences,  SMS  workshops,  and  process  definition.): 


SEPG  AIS  Support 


FSA/DSE 

Total 

Manaoers 

Members 

Practitioners 

Division 

Other 

FSACL 

3 

3 

3.2 

2.75 

FSACO 

3.73 

3.33 

3 

2.83 

2  DK/NA 

1  DK/NA 

- 

FSADE/ 

DSE-MP 

2.75 

2.4 

2.2 

2.9 

3  DK/NA 

2.8 

FSAIN 

2.82 

3.38 

4DK/NA 

2. 

1  DK/NA 

2.96 

7  DK/NA 

2.43 

2  DK/NA 

2.4 

3  DK/NA 

FSAKC 

3.12 

2.67 

1  DK/NA 

2.5 

3.48 

5  DK/NA 

2.8 

2  DK/NA 

2.67 

1  DK/NA 

FSAPE 

3.0 

3.0 

4.0 

2  DK/NA 

2.25 

2  DK/NA 

4.0 

2  DK/NA 

Comments 

•  As  an  organization,  it  seems  we  focused  too  much  on  the  SMS  and  should  have 
focused  on  the  CMM  sooner. 

•  There  seemed  to  be  a  communication  gap  in  getting  information  from  the  various 
FSO  SPI  conferences,  etc.,  to  them. 

•  The  value  of  the  SPI  conferences  was  questioned  when  the  main  agenda  was  status 
reports  versus  sharing  solutions,  setting  direction,  resolving  issues.  The  VTC  is  an 
excellent  format  for  status  reports.  However,  face-to-face  is  more  beneficial  when 
synergy  is  required  (i.e.,  setting  direction,  alternative  approaches,  and  resolving 
issues/obstacles). 

•  Although  I  only  experienced  some  training  plus  one  SPI  conference,  I  know  that  there 
was  plenty  of  support  activities  for  SPI  by  FSO. 
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•  Process  definition  activities  of  TWGs  did  not  provide  for  equal  input  by  all  participants 
when  they  were  initially  advertised  as  such.  FSO  TWG  participants  did  not  contribute 
meaningfully  to  TWG  process  definition  activities. 

•  T raining  provided  was  fine.  Support  and  process  definition  were  very  sketchy  at 
times  and  not  very  conscientious. 

•  I  would  have  to  say  that  support  shown  by  the  FSO  was  extremely  high  considering 
the  training  dollars  allocated. 

•  Poor  direction  and  metrics  and  FPA  leads  to  useless  work  and  data  that  in  some 
cases  we  know  is  meaningless.  Important  one  day  and  not  the  next.  Adds  more 
overhead. 

•  Only  the  training  that  was  contracted  was  worthwhile.  T raining  on  the  tools  and  on 
the  individual  KPAs  was  poor. 

•  FSO  needs  to  take  a  more  team  approach,  not  dictatorial. 

•  The  process  definition  efforts  were  marginal  at  best.  Processes  were  dictated  rather 
than  created  with  the  input  of  all  FSAs. 

•  Reading  slides  and  the  SMS  to  software  engineers  is  insulting.  SPI  conferences 
shared  little  useful  information  for  their  costs. 

•  Felt  we  were  being  “force  fed”  processes/lools. 

•  Other  than  selected  members  of  the  Steering  Group,  I  really  felt  we  were  pretty  much 
on  our  own. 

•  Not  involved 

•  Gini  did  great  -  miss  SEPG  newsletters  -  Anita  did  great  job  on  maintaining  and 
soliciting  feedback  on  SMS  updates 

•  Never  attended  any  of  these  except  training 

•  Never  attended  any  of  these  and  did  not  know  they  were  available 

•  Would  have  done  better  if  each  FSA  pursued  Level  2  on  own 

•  Not  sure  that  I  have  been  involved  with  these  FSO  activities 

•  I  have  not  received  any  training  or  support  of  this  nature 

•  Look  at  the  committees  charged  with  developing  SPI  materials 

•  No  money  for  training  or  attendance 

•  I  have  had  unit  test,  SQA,  and  PM  -  one  exceptionally  good  contractor  instructor  - 
keep  this  man  -  have  not  been  to  conferences  or  workshops 

•  Non-existent 

•  Definitely  went  extra  mile  on  training  -  we  had  many  individual  training  courses  and 
special  class  prior  to  rating  -  class  was  tailored  for  our  system  and  was  very  helpful 

•  Really  hard  to  judge  because  of  the  difficulty  of  the  tasks 

•  Haven’t  attended  any  for  a  long  time' 

•  Except  for  basic/overview  training,  who  gets  to  attend/participate  in  their  activities 

•  Support  is  good  but  could  be  better  if  classes  were  given  more  often  and  were  not  so 
long  -  ideal  is  1.5  hours 

•  My  experience  here  is  limited  to  formal  training  that  was  funded  by  the  FSO 

•  SMS  is  rarely  talked  about  until  an  impending  meeting,  then  it  is  the  most  important 
issues  we  have 

•  Classes  and  booklets  were  given  to  everyone  in  the  division 

•  At  first,  FSO  was  active  -  now  dropped  out  of  sight 

•  I  have  not  been  involved 

•  Not  directly  involved 

•  The  training  needs  to  be  geared  around  system  specific  processing 

•  What  FSO  activities?  None  to  speak  of  -  when  they  did  come  in,  it  was  as  dictators 
and  giving  wrong  information  -  FSO  lost  credibility  in  many  people’s  minds 

•  Over-definition  of  some  processes,  not  enough  on  others  (policies  for  Web,  MTMO, . 
oracle,  etc.)  would  prefer  leadership  and  strategic  planning  to  micromanaging 

•  Good  at  first,  but  seems  to  be  losing  steam. 
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•  These  were  very  helpful  to  SEPG  members,  but  not  sure  how  valuable  to 
practitioners. 

•  FSO  provided  leadership  and  budget  support  for  the  SPI  program.  Without  this 
support,  very  little  would  have  been  done. 

•  Training  was  mostly  to  explain  terminology  and  “read”  the  KPAs,  not  “how  to”  on 
development  of  policies/procedures. 


9.  Please  rate  the  tools  provided  by  FSO  for  SPI  Initiatives  (Tools  include  LRS,  CMIS,  and 
SMS  Survey  Tool.): 


FSA/DSE 

Total 

Manaaers 

SEPG 

Members 

AIS 

Practitioners 

Support 

Division 

Other 

FSACL 

2.75 

2.67 

3 

3 

2.25 

FSACO 

2.79 

3 

3 

2.57 

3 

- 

FSADE/ 

2.16 

2.09 

1.83 

2.24 

2 

DSE-MP 

FSAIN 

2.64 

2.44 

2. 

1  DK/NA 
2.96 

2.25 

2.42 

FSAKC 

2.70 

3  DK/NA 
1.86 

1  DK/NA 
1.75 

3  DK/NA 
3.07 

1  DK/NA 
3.00 

1  DK/NA 
2.00 

FSAPE 

3.41 

3.25 

4.25 

1  DK/NA 
3.00 

2  DK/NA 
3.33 

1  DK/NA 

Comments 

•  FSO  SPI  Tools  help  get  the  job  done  but  are  not  user  friendly  and  can  be 
cumbersome. 

•  Overall  approach  to  Employing  support  tools  seemed  disjointed.  The  tools  with  the 
least  support  are  CMIS  and  the  SMS  Survey  Tool.  It  appears  that  what  people  want 
is  an  integrated  tool  set  for  each  of  the  functions.  For  instance: 

a)  A  single  integrated  tool  set  that  performs  CM  control/migration  and  status 
accounting,  and  Cl  inventory.  CMIS  does  only  2  of  these. 

b)  A  single  integrated  tool  set  that  helps  track  hours  and  other  project  management 

activities. 

A  possible  alternative  is  to  purchase  COTS  that  are  somewhat  tailorable. 

•  Need  to  purchase  standard  CM,  test,  and  project  estimation  tools. 

•  Concerned  that  tools  being  used  have  not  all  implemented  and  reached  CMM  Level 
2.  CMIS  could  not  provide  a  CM  plan,  for  example. 

•  LRS  and  CMIS  are  good  tools  to  use.  The  Survey  Tool  was  basically  useless  a  year 
ago  when  it  was  needed. 

•  Ido  not  use  CMIS.  LRS  is  fine.  SMS  survey  tool  I  cannot  recall  what  this  was. 
Again,  the  tools  duplicate  processes  like  estimating  and  PM  activities  and  are  not 
necessarily  designed  for  the  AIS. 

•  LRS  is  very  prone  to  breakdown.  The  SMS  survey  tool  became  less  useful  as  the 
focus  shifted  from  SMS  to  the  CMM  compliance. 

•  Sometimes  the  release  of  the  tools  to  accomplish  SPI  initiatives  is  difficult  to 
determine. 

•  SMS  Survey  Tool  is  the  most  beneficial  of  these  tools;  however,  the  tool  would  be 
more  useful  if  it  would  be  based  entirely  on  the  CMM. 

•  Tools  are  very  poor.  If  we  are  to  proceed  with  this  effort,  we  are  definitely  in  need  of 
additional  automated  tools.  We  cannot  be  expected  to  support  a  Level  2  or  3 
organization  with  archaic  tracking  devices. 

•  AISs  expected  to  adopt  to  tools  developed  for  AISs  in  different  environments  (i.e., 
CMIS).  Trying  to  force  round  pegs  into  square  holes.  Attempts  to  use  LRS  for 
metrics  in  ways  it  is  inadequate  for. 
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•  LRS  is  the  best  of  the  bunch.  CMIS  was  a  bad  decision  based  on  a  lack  of 
knowledge  of  what  was  needed.  It  was  proven  not  to  be  the  universal  tool  as  was 
advertised.  The  SMS  Survey  Tool  provided  no  benefit. 

•  CMIS  =  poor  performance/not  adaptable  to  large  systems. 

•  These  tools  are  poorly  integrated,  difficult  to  use,  and  are  an  insult  to  us  as  a 
software  development  organization. 

•  They  all  have  that  awkward,  home-grown  feel.  Probably  would  have  been  better  to 
find  tools  that  suit  the  needs  rather  than  enforcing  tools  to  fit  needs  that  were  never 
envisioned  when  they  were  developed. 

•  LRS  is  probably  the  best  of  a  bad  bunch.  Has  anyone  besides  me  been  appalled  at 
the  amount  of  money  this  place  has  spent  on  CMIS?  We  should  have  bought  COTS, 
then  built  the  SCM  function  around  it,  instead  of  having  everyone  keep  trying  to 
change  the  tool  to  match  existing  practices. 

•  CMIS  is  old  and  outdated.  Written  by  individuals  for  specific  site  and  everyone  is 
expected  to  use.  CCB  process  has  broken  down.  LRS  is  not  designed  for  a  large 
organization.  Lots  of  problems  with  software. 

•  LRS  and  CMIS  should  talk  to  each  other  and  the  SMS  survey  tool  was  cumbersome 
and  an  exercise  in  futility  of  status  reporting  for  no  purpose  other  than  HQ  had  a  tool! 

•  It  appeared  CMIS  tool  was  picked  without  study  of  methodologies  at  sites. 

•  PANAPT  is  best. 

•  It  seems  like  a  lot  to  fill  out  to  keep  track  of  what  you  are  doing. 

•  LRS  for  entering  time  and  don’t  use  CMIS 

•  Could  have  used  more  definition  on  overhead  categories  -  SMS  survey  good  to 
maintain  checklist  when  first  implementing  -  could  not  load  new  version  as  too  big  - 
CMIS  helpful  but  need  tools  for  TDR  status,  version  control,  sizing,  library  controls 

•.  CMIS  needs  something  like  a  tiger  team,  not  endless  meetings,  in  order  to  implement 

•  Input  correct  time  into  LRS,  then  maybe  we  can  use  it  as  a  tool  to  measure  progress 
-  CMIS  needs  a  library  system  to  be  effective 

•  Only  training  received  was  on  CMIS 

•  LRS  times  not  entered  correctly  to  provide  accuracy  -  CMIS  only  as  good  as  data  - 
better  if  linked  to  library  system/LRS 

•  CMIS  very  good  -  LRS  sufficient  -  survey  tool  inadequate 

•  CMIS  is  nice  report  tracking  tool  but  just  adds  duplicate  effort  and  does  not  control 
software  movement  and  versions 

•  Only  involved  with  CMIS  to  limited  extent  -  it  controls  documentation  and  approvals 
but  does  nothing  for  physical  configuration  management 

•  Have  only  taken  CMIS  training  and  not  using  it  as  part  of  my  duties  at  this  time 

•  What  do  the  tools  names  have  to  do  with  SPI?  We  need  to  implement  Designer 
2000  as  a  process  tool 

•  LRS  is  good  -  No  CMIS  training  -  SMS:  next  version  should  be  much  shorter 

•  Multiple  data  entry  is  insufficient 

•  We  have  LRS  -  we  use  OSEDT,  not  CMIS 

•  LRS:  eeny,  meeny,  miny,  mo,  what  should  I  charge  the  customer  for  today?  CMIS: 
“user  friendly”  not  an  attribute  of  this  exercise  in  futility.  SMS  Survey  Tool:  Have  not 
seen  nor  do  I  know  what  it  is/does. 

•  LRS  and  CMIS  are  poor  -  data  collected  may  be  useful  at  FSO  but  are  of  no  worth  to 
me  -  both  impractical.  SMS  OK 

•  New  LRS  pending  -  has  been  delayed  again.  CMIS  is  not  functioning  yet. 

•  Never  see  anything  happen  because  of  surveys.  Not  using  CMIS  yet.  LRS  is  OK  but 
not  very  useful.  Need  report  that  will  give  summary  total  of  how  many  hours  were 
against  a  specific  SCR. 

•  These  tools  were  very  much  needed  and  make  our  jobs  easier 

•  CMIS  could  be  improved 

•  OSEDT  is  a  great  tool  -  LRS  seems  to  be  very  good  also. 
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•  On-line  (LAN)  tools  are  used  by  everyone  in  the  division  -  LRS,  CMIS,  Word,  Excel, 
ccmail  are  a  few  tools  used 

•  They  helped 

•  CMIS  very  good 

•  SMS  and  LRS  are  functional  for  what  they  are  designed  to  do 

•  We’ve  had  CMIS  training,  but  haven’t  started  to  use  it  yet 

•  LRS  is  hard  to  use  -  New  CMIS  is  needed  -  Training 

•  LRS  and  CMIS  proved  invaluable.  The  survey  tool  provided  little  value 

•  LRS  is  used,  and  it  is  on  the  network. 

•  Does  not  provide  what  we  need  at  the  developer’s  level 

•  CMIS  does  not  add  value  to  our  processes  but  makes  them  more  cumbersome 
because  of  additional  procedures 

•  Couldn’t  use  CMIS  due  to  inadequate  environment.  LRS  cumbersome  and  needs  to 
be  more  user-friendly.  Data  not  always  accurate. 

•  Still  trying  to  get  there. 

•  We  were  tasked  to  perform  mission  with  tool  incapable  of  meeting  requirements 

•  LRS  is  a  joke  -  no  one  enters  correct  information  for  it  to  be  of  any  value.  CMIS  is 
okay.  SMS  survey  tool  is  a  joke. 

•  You  must  be  joking  -  not  user  friendly  -  not  compatible  -  no  interface  -  redundant 

•  LRS  has  been  co-opted  by  business  management  people  and  is  virtually  useless  as 
a  source  for  software  metrics. 

•  Both  LRS  and  CMIS  were  significant  factors  in  our  achieving  Level  2.  These  tools 
provided  cost  and  effort  data  as  well  as  manage  and  teach  the  requirements. 

•  All  three  of  these  tools  were  extremely  useful,  if  used,  to  assist  in  managing  and 
tracking. 

•  Tools  are'  valuable  but  there  needed  to  be  more  standardization  across  FSO  for 
usage. 

•  CMIS  vital  for  CM  portion. 

•  They  do  not  provide  for  cross-walking  as  each  is  structured  so  differently. 

•  LRS  and  CMIS  were  part  of  our  practice  before  SPI  came  along.  I  don’t  know  what 
the  SMS  Survey  Tool  is. 

•  Familiar  with  tools,  but  not  with  FSO  support. 

•  Most  tools  used  are  standalone.  A  greater  improvement  to  the  software  process 
would  be  a  more  integrated  unit.  There  is  no  link  between  CMIS  and  LRS.  There  is 
no  link  for  CMIS  and  our  Version  Control  System.  Each  piece  acts  alone. 

•  LRS  and  CMIS  are  Very  Good.  SMS  is  Inadequate. 


10.  Please  rate  the  local  SEPG  support  you  received  on  SPI  Initiatives; 


FSA/DSE 

Total 

Manaaers 

SEPG 

Members 

AIS 

Practitioners 

Support 

Division 

Other 

FSACL 

3.94 

4 

4.25 

3.8 

3.75 

FSACO 

3.85 

4.33 

4.5 

3.43 

4 

- 

FSADE/ 

DSE-MP 

2.89 

2.81 

2.8 

3.0 

2  DK/NA 

2.33 

FSAIN 

2.96 

3.4 

2DK/NA 

2.0 

1  DK/NA 

3.24 

4  DK/NA 

2.14 

1  DK/NA 

2.5 

3  DK/NA 

FSAKC 

4.07 

4.29 

4.00 

3  DK/NA 

4.2 

3  DK/NA 

3.71 

3.33 

1  DK/NA 

FSAPE 

4.00 

3.75 

5.00 

4.0 

3  DK/NA 

4.0 

1  DK/NA 
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11.  Comments  on  local  SEPG  support: 

•  SEPG  was  helpful  in  implementing  SPI  initiatives 

•  Wide  range  -  from  ‘absolutely  no  value’  to  ‘only  reason  the  organization  made  any 
progress’ 

•  Need  to  ensure  the  SEPG  is  knowledgeable,  experienced  and  can  give  examples  of 
mechanisms  to  the  AISs.  The  “standard”  mechanisms  could  come  from  either  the 
site  or  FSO  level,  but  people  need  to  have  samples  of  mechanisms  to  translate  the 
CMM  into  workable  processes.  The  lessons  here  may  be: 

a)  Formulating  and  implementing  a  training/education  agenda  for  SEPG 

b)  Network  frequently  with  other  organizations  implementing  SPI,  in  order  to  share 

approach,  lessons  learned,  and  sample. 

c)  Obtain  consultants  periodically  at  key  points  in  implementing  the  SPI  program 
for 

both  the  FSO-HQ  and  local  site  levels. 

•  CM  and  SQA  provide  excellent  support  for  AIS  and  for  SEPG  members. 

•  All  local  SEPG  members  were  very  helpful  and  informative.  They  were  always  willing 
to  help  research  the  ahswers  to  questions. 

•  They  are  very  helpful. 

•  In  some  areas,  the  support  was  exceptional,  but  others  it  was  marginal. 

•  The  SEPG  is  very  well  run  and  members  are  knowledgeable  about  their  areas  of 
expertise.  If  asked,  they’ve  provided  valuable  information  to  all. 

•  SEPG  has  assisted  in  crosswalks  review  of  implementation  plans  and  assessments. 

•  Again,  this  is  somewhat  reflective  of  the  FSO  support  received. 

•  I  believe  that  the  SEPG  group  was  instrumental  as  far  as  serving  as  a  liaison 
between  the  FSO  and  the  FSA. 

•  Sometimes  overzealous  in  requiring  artifacts  that  have  no  value  to  the  AIS.  Apply 
procedure  for  the  sake  of  procedure  regardless  of  any  benefit. 

•  I  don’t  think  that  the  SEPG  has  ever  realized  that  a  group  is  as  powerful  as  people  let 
it  be.  For  example,  if  they’d  have  sent  out  a  few  directives  “from  now  on,  everyone 
must ...”  instead  of  trying  to  get  everyone  to  embrace  their  philosophy,  they’d 
probably  gotten  a  lot  of  stuff  implemented.  As  it  is.  people  feel  they  had  a  choice  and 
some  chose  no. 

•  The  local  SEPG’s  support  was  the  key  to  the  successful  SPI  implementation  at 
FSAKC. 

•  Our  SEPG  was  ALWAYS  there  to  solve  problems  and  answer  questions. 

•  Very  hard-working  and  conscientious. 

•  Without  them,  there  is  no  Level  2. 

•  Can’t  say  enough  good  things  about  our  SEPG  Team. 

•  It  seemed  to  me  the  SEPG  assisted  SMD  personnel.  I  didn’t  realize  until  later  this 
was  actually  a  SEPG  function.  Also,  I  am  not  aware  of  all  the  efforts  of  the  SEPG, 
only  those  that  directly  involved  me. . 

•  I  get  SPI  news  almost  every  month  and  Word  of  Week.  SEPG  does  excellent  work. 
Make  better  organization  for  designers,  programmers,  and  testers.  Saves  US 
Government  a  lot  of  money. 

•  Our  SEPG  Team  has  made  a  valiant  effort,  but  had  only  received  a  lukewarm 
welcome. 

•  Could  be  more  proactive,  leading  the  way,  bringing  the  practitioner  needs  and  ways 
to  meet  them  to  management,  rather  than  limiting  their  efforts  to  what  they  feel  they 
can  talk  management  into. 

•  Once  AIS  is  educated,  SEPG  should  act  as  consultants,  not  implementation  project 
managers.  SEPG  needs  to  attend  conferences  and  spread  new  ideas  for  support. 

•  lam  not  familiar  with  the  SEPG. 

•  Don’t  have  an  SEPG  anymore. 

•  This  group  needs  more  PR  to  let  the  rest  of  the  world  know  who/what  they  are/do. 
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•  No  visible  SEPG. 

•  Early  on,  they  treated  software  engineers  as  if  inferior.  Took  credit  for  work  they  did 
not  do. 

•  I  have  not  received  any  local  SEPG  support. 

•  Support  is  lacking. 

•  None. 

•  Not  aware  of  SEPG  support.  Just  started  working  with  SPI/KPA  teams. 

•  I  am  too  far  down  the  chain  of  command  to  have  contact. 

•  What  SEPG? 

•  I  consider  them  overhead. 

•  Occasionally  spread  too  thin  and  occasionally  impractical  with  uncooperative  attitude. 

•  Always  ready  to  answer  questions  and  to  do  what  they  can  to  help  us  understand 
and  set  goals  for  SPI. 

•  Any  questions  are  answered  in  this  area. 

•  Not  much  too. 

•  Unknown. 

•  Our  division  reps  keep  us  very  well  informed  about  SPI  activities. 

•  I  don’t  know  what  SEPG  is. 

•  More  SEPG  support  will  be  helpful.  They  should  closely  monitor  our  practices  and 
give  us  guidance  on  what  improvements  need  to  be  made. 

•  Very  supportive. 

•  Not  properly  staffed. 

•  What  does  SEPG  stand  for? 

•  No  contact 

•  There  has  been  little  or  no  support  shown 

•  Provided  support  when  requested 

•  Good  effort  but  failed 

•  Never  supportive.  Always  dictatorial  and  combative 

•  No  SEPG  -  inadequate  when  we  did  have  one.  The  process  has  become  a  total  joke 
in  our  FSA.  The  other  FSAs  are  doing  it  better.  Ours  has  done  a  complete 
turnaround  on  our  vision  and  methods. 

•  SEPG  is  currently  understaffed  and  undertrained  through  no  fault  of  their  own.  Goes 
back  to  management  (FSO/FSA)  support. 

•  Local  SEPG  provided  support  as  requested  to  project  SPI  staff. 

•  I  thought  the  support  was  quite  good,  once  an  AIS  was  receptive  to  it. 

•  All  members  were  very  knowledgeable  and  cooperative.  They  continued  to  go  out  of  their 
way  to  facilitate  a  project’s  improvement. 

••  We  have  a  very  knowledgeable  and  helpful  SEPG.  They  have  assisted  me  and  provided 
guidance  concerning  procedures. 

12.  Please  rate  the  effectiveness  of  the  SCE  process: 


FSA/DSE 

Total 

Manaaers 

SEPG 

Members 

AIS 

Practitioners 

Support 

Division 

Other 

FSACL 

3.13 

2.67 

3.5 

3 

3.25 

FSACO 

3.43 

3.67 

3.5 

3.14 

4 

- 

FSADE/ 

3.45 

3.8 

3.5 

3.41 

2.67 

DSE-MP 

FSAIN 

3.91 

4.17 

3.5 

3DK/NA 

4.2 

3.67 

2.57 

FSAKC 

3.48 

2.86 

4.0 

3.70 

3DK/NA 

3.43 

6DK/NA 

2.67 

FSAPE 

3.94 

3.5 

4.25 

5DK/NA 

4.00 

4.00 

1  DK/NA 
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Comments: 

•  Most  comments  centered  on  the  accuracy  of  the  assessments  and  the  help  provided 
by  the  final  findings  reports.  The  assessments  were  viewed  as  professional,  fair,  and 
accurate  even  when  the  findings/results  did  not  result  in  a  CMM  Level  2  rating. 

•  Could  have  been  better  and  more  consistent  if  the  process  had  been  documented. 

•  The  SCE  team  members  were  very  professional.  It’s  believed  that  the  SCE  process 
was  not  uniformly  applied  between  FSAs. 

•  I  do  not  feel  the  SCE  criteria  have  been  applied  consistently.  Credibility  is  lacking. 

•  Never  been  involved  in  actual  SCE 

•  What  is  an  SCE? 

•  Do  not  know  what  “SCE”  is 

•  What’s  the  SCE  process? 

•  What  is  SCE? 

•  Not  done  yet 

13.  Please  rate  how  well  the  SCE  findings  helped  in  action  planning  for  future 
irnprovements: 


FSA/DSE 

Total 

Manaoers 

SEPG 

Members 

AIS 

Practitioners 

Support 

Division 

Other 

FSACL 

3.56 

3 

4 

3.6 

3.5 

FSACO 

3.31 

4.33 

3.5 

3.5 

1  DK/NA 

3.5 

FSADE/ 

DSE-MP 

3.64 

3.91 

3.6 

3.59 

7  DK/NA 

3.33 

FSAIN 

3.85 

4.17 

3.5 

4.11 

3.75 

5  DK/NA 

2.43 

6  DK/NA 

FSAKC 

3.47 

3.00 

3.75 

3.73 

6  DK/NA 

3.29 

2.67 

1  DK/NA 

FSAPE 

3.81 

3.25 

4.0 

3.8 

1  DK/NA 

4.33 

Comments: 

•  Most  comments  centered  on  the  accuracy  of  the  assessments  and  the  help  provided 
by  the  final  findings  reports.  The  assessments  were  viewed  as  professional,  fair,  and 
accurate  even  when  the  findings/results  did  not  result  in  a  CMM  Level  2  rating. 

•  Answer  relates  only  to  improvements  relating  to  achieving  Level  2. 

•  Not  evaluated  yet 

•  Identified  weak  areas 

•  Don’t  know  what  “SCE”  is 

•  Not  done  yet 

14.  Comments  on  assessments  and  related  activities: 

•  SCE  tied  everything  together  and  helped  identify  areas  needing  improvement 

•  SCE  findings  were  too  vague 

•  Most  comments  centered  on  the  accuracy  of  the  assessments  and  the  help  provided 
by  the  final  findings  reports.  The  assessments  were  viewed  as  professional,  fair,  and 
accurate  even  when  the  findings/results  did  not  result  in  a  CMM  Level  2  rating. 

•  The  assessments  were  helpful  in  identifying  weak  areas  in  our  processing.  The  IP 
was  helpful  in  getting  an  insight  of  how  the  workers  viewed  the  process. 
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•  The  lessons  learned  were  helpful  for  the  next  projects.  SCE  process  should  have 
been  documented  to  provide  more  consistent  evaluations.  The  success/failure 
perception  was  overemphasized. 

•  The  SCE  pointed  out  the  same  areas  of  improvement  which  we  knew  were  weak.  I 
think  the  SCE  should  be  done  by  outside  people,  outside  the  FSA,  not  by  peers  at 
other  centers. 

•  Current  teams  are  working  well. 

•  The  assessments  are  a  good  measure  of  implementation  progress. 

•  Some  inconsistency  among  the  team  members  possibly  due  to  misunderstanding  of 
the  actual  organization’s  process  or  possible  pressure  from  other  team  members  to 
rate  otherwise. 

•  The  assessments  were  very  helpful  in  evaluating  our  progress  and  in  preparation  for 
the  SCE. 

•  Credibility  is  not  what  it  should  be. 

•  The  SCE  findings  were  very  accurate  and  quite  useful  in  developing  programs  to 
change  processes. 

•  It  was  extremely  frustrating  to  go  through  the  process,  for  the  extended  visit  and  have 
the  team  decide  they  could  not  give  us  a  rating.  Not  apparent  guidance  when  a 
stalemate  occurs  between  members  of  an  evaluation  team. 

•  While  painful  at  times,  the  SCEs  drove  home  the  CMM  concepts  and  helped  pull  the 
FSA  together.  Because  the  results  were  focused  at  Level  2,  the  findings  only 
addressed  small  improvements. 

•  Don’t  think  they  were  done  fairly  across  the  board.  Some  systems  appeared  to  be 
under  tighter  scrutiny  than  others. 

•  The  “pass  or  die”  attitude  wasn’t  the  fault  of  the  SCE  process.  There  were  hard 
feelings  about  the  SCEs.  The  concept  of  having  in-house  assessors  was  good,  but 
in  practice  it  was  a  losing  situation  -  unless  the  site  passed  -  then  you  could  be  pals. 
How  many  sites  followed  their  “recommendations  for  improvement”  to  implementation 
once  they  got  Level  2?  Findings  were  good,  but  I  doubt  the  level  of  follow-through. 

•  Interpretation  of  some  CMM  elements  are  vague  and  leaves  too  much  discretion  to 
the  assessment  team  members. 

•  Need  to  have  time  to  operate  at  Level  2  and  iron  out  procedures/processes  before 
proceeding  to  Level  3. 

•  They  take  too  much  time  and  resources  if  they’re  only  to  be  used  to  establish  a 
grade.  They’d  be  very  worthwhile  if  they  were  used  to  focus  and  plan  improvement 
efforts. 

•  Still  waiting  for  this  assessment.  Questionnaire  should  have  been  postponed  60 
days. 

•  Can  feel  political  and  competitive  at  it’s  worst.  Can  feel  worthwhile  and  instructive  at 
its  best. 

•  The  findings  I  read  were  more  about  the  SMS,  not  CMM. 

•  Headed  in  right  direction,  but  still  a  lot  to  do. 

•  Dates  set  for  an  AIS  evaluation  are  often  set  outside  the  AIS  and  do  not  consider 
need  to  implement,  train,  improve  and  institutionalize  the  processes  for  the  AIS. 

•  The  extra  forms,  steps,  and  procedures  should  add  much  cost  and  waste  to  work 

•  Very  good  for  efforts  to  be  applied 

•  None  of  constructive  value 

•  These  are  enlightening  and  beneficial  -  Determine  the  extent  of  the  SPI 
understanding 

•  Assessments  of  immeasurable  value 

•  SCE  helped  me  identify  where  I  was  lacking  knowledge  of  the  process,  so  I  could 
work  on  improving  my  awareness  in  these  areas 

•  The  first  assessment  was  a  little  confusing  as  to  what  was  needed.  The  second  one 
was  much  better. 
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•  Weaknesses  were  pointed  out  and  explained. 

•  The  questions  sometimes  seem  vague  and  obscure.  Maybe  they  need  more 
explanation,  or  to  use  fewer  unknown/forgotten  acronyms 

•  No  SCE  yet 

•  Some  questions  on  SCE  were  value  -  more  probably  misinterpreted 

•  Pointed  out  weaknesses  for  you  to  improve  upon  before  next  assessment 

•  Question  qualification  and  experience  of  assessors  -  Prefer  to  contract  out  to  an 
organization  experienced  in  assessments 

•  Too  many  of  the  assessors  had  hidden  agendas.  Revenge,  getting  even,  etc. 

•  Assessments  by  trained  personnel  are  fair  and  objective.  Much  goes  into  the 
process.  Strengthens,  and  weaknesses  are  clearly  defined.  Other  information 
provided  for  future  improvement. 

•  Assessment  was  conducted  in  an  impartial  and  professional  manner. 

•  Assessment  was  a  stressful  time. 

•  Assessments  helped  set  direction  and  identify  soft  areas  in  our  software  engineering 
processes. 

•  Provided  areas  where  improvements  could  be  made. 

•  Very  thorough. 

•  Assessment  was  very  fair. 

•  They  have  been  conducted  in  a  professional  manner.  It  is  a  very  good  practice  to 
interview  workers  separate  from  managers. 

15.  Please  rate  the  usefulness  of  the  System  Modification  Scenario  as  a  process 
definition: 


FSA/DSE 

Total 

Manaaers 

SEPG 

Members 

AfS 

Practitioners 

Support 

Division 

Qther 

FSACL 

2.56 

2 

2.5 

2.8 

2.75 

FSACO 

2.92 

2.67 

.3 

2.86 

3.5 

- 

FSADE/ 

DSE-MP 

2.56 

1.9 

1.67 

2.83 

1  DK/NA 

2.67 

FSAIN 

3.07 

3.17 

2. 

1  DK/NA 

3.17 

1  DK/NA 

2.88 

1  DK/NA 

2.91 

2  DK/NA 

FSAKC 

3.16 

2.17 

1  DK/NA 

2.00 

3.58 

4  DK/NA 

3.43 

2.67 

1  DK/NA 

FSAPE 

3.21 

3.0 

3.25 

4.33 

3  DK/NA 

2.33 

Comments: 

•  SMS  is  too  detailed 

•  SMS  is  oriented  to  another  site’s  process  and  doesn’t  impact  daily  work 

•  SMS  is  too  bulky,  complicated,  difficult  to  read  and  follow.  It  should  have  more 
graphics.  Templates  for  various  mechanisms  (SDP,  SCM  Plan,  SQA  Plan,  status 
meeting  agenda/minutes,  checklists  for  SQA  and/or  CM  audits,  etc.)  would  have 
been  helpful,  but  not  the  detailed  expansive  how-to  narrative  procedures.  It 
appeared  to  be  too  detailed  where  it  wasn’t  needed  (how  to  set  up  a  meeting)  and 
not  detailed  enough  where  it  was  needed  (what  should  be  done  in  a  baseline  audit, 
what  should  be  the  outputs  -  even  a  sample  of  minimum  information  on  a  template). 

•  Conflicting  direction  on  what  had  to  be  implemented  in  the  SMS  (i.e.,  task  level 
versus  procedure  level)  was  frustrating  and  time-consuming.  Most  respondents’ 
comments  agreed  that  task  level  (the  what)  is  the  appropriate  level  for 
implementation.  Lessons  here  may  be  getting  more  consultants  early  and  at  critical 
SPI  program  milestones  at  both  an  FSO-HQ  and  site  level  to  help  guide  the  direction 
and  point  out  pitfalls  to  avoid. 
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•  Sufficient  for  projects  to  use  as  a  template  to  tailor  their  own  process. 

•  I  think  the  SMS  is  useful  as  a  process  definition  for  new  development.  It  is  not  useful 
as  a  process  definition  for  modification. 

•  I  believe  that  the  SMS  is  adequate  for  new  development,  but  for  maintenance,  it  is 
overkill. 

•  Had  the  communication  and  planning  by  FS  senior  management  been  better  up 
front,  the  product,  SMS,  would  have  been  better.  Also,  if  the  various  FSA  TWG 
members  truly  had  equal  input,  the  product  would  have  been  better  received. 

•  The  SMS  is  a  good  policy  guideline,  but  since  the  assessments  are  based  on  the 
CMM,  it  is  rather  redundant  or  confusing.  Procedures  are  totally  tailored  in  any  case; 
therefore,  the  SMS  is  generally  superfluous. 

•  Adherence  to  the  SMS  resulted  in  consistent  failure.  Inadequacies  relate  directly  to 
survey  (question  #1). 

•  Despite  the  fact  that  we  are  one  “FSO,”  we  are  many  different  systems  (size, 
platforms,  sites)  with  many  different  customers.  It  is  very  difficult  to  put  this  in  one 
book. 

•  I  consider  the  SMS  sufficient  as  a  starting  point.  To  some,  areas  of  the  SMS  were 
misleading  -  example,  estimation. 

•  It’s  fine  as  an  overall  FSO  policy,  but  not  quite  as  useful  as  a  software  process 
definition  tool.  The  CMM  better  defines  the  necessary  goals  and  should  be  stressed 
more. 

•  The  SMS  is  an  extremely  detailed  manual.  In  the  early  stages,  it  was  quite  confusing 
as  to  which  manual  to  use  in  pursuing  Level  2  objectives  (SMS  versus  CMM). 

•  Goes  way  overboard  on  creating  an  inadequate  a  mount  of  overhead  which  does  not 
enhance  or  add  value  to  the  system  goals.  Too  rigid  in  requiring  too  much. 

•  Too  hard  to  follow,  repetitive,  and  inadequate  in  reaching  Level  2.  Some  parts  were 
useful  but  as  a  whole  the  document  went  into  too  much  detail. 

•  Some  pieces  of  the  SMS  were  useful  in  improvement  activities  once  you  could  find 
them.  The  format  structure  and  inflexible  nature  of  the  document  made  it  unwieldy 
and  practically  impossible  to  use.  An  organizational  process  must  be  flexible  enough 
for  each  project  to  tailor  it  to  its  specific  needs,  customers,  etc. 

•  Not  bad  to  use  as  a  starting  point  or  template,  but  still  required  a  lot  of  other  work  to 
get  to  Level  2.  Too  many  CMM  areas  not  covered.  NOT  USER  FRIENDLY! 

•  As  with  anything  written  by  a  lot  of  people,  then  compiled  together,  the  document  is 
uneven  -  some  places  have  too  much  detail  -  others  not  enough.  It’s  better  than  its 
reputation  would  suggest,  but  I  find  it  crirninal  that  FSO  ignored  the  problem  that 
irriplementing  the  SMS  did  not  make  the  site  CMM  Level  2. 

•  The  SMS  is  difficult  to  follow,  is  too  procedurally  based,  and  tries  to  mandate 
standardization. 

•  Lots  of  good  information,  not  the  easiest  document  to  read. 

•  The  SMS  tended  to  get  too  far  into  the  day-to-day  operations  on  some  areas.  This 
appeared  to  stem  from  the  SMS  author  FSA’s  frame  of  reference  to  their  local  system 
and  didn’t  seem  to  take  the  basics  into  account  such  as  the  difference  in  size  of  a 
local  or  FSO-wide  AIS  versus  service-specific  pay  related  systems. 

•  Too  big,  bulky,  way  too  much  detail.  Outline  the  intended  function  and  the  results 
you  desire  (CM  -  version  control,  etc.)  and  then  step  back  and  let  the  individual 
activities  fulfill  these  processes  according  to  their  specific  needs.  Audit  periodically 
by  FSO. 

•  At  first,  seemed  painful  and  redundant.  After  working  with  it,  could  see  the  elegance. 
Good  procedures  model,  but  will  not  get  an  AIS  versed  in  CMM  or  to  Level  2. 

•  “PTR”  in  SMS  should  be  changed  to  Program  T rouble  Report  to  match  CMIS 
terminology. 

•  It  needs  to  be  flexible.  Not  every  project  is  the  same.  What  works  for  one  may  not 
be  cost-effective  for  another. 

•  Haven’t  worked  with  this  much. 
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•  FSO/FSA  management  focused  too  much  on  SMS  for  reaching  Level  2.  Lost  site  of 
CMM.  Evaluations  against  CMM. 

•  Could  have  developed  o\wn  procedures  faster. 

•  Too  repetitious.  Too  cut  and  dried.  Leaves  no  room  for  tailoring  to  the  AIS.  Seems 
to  put  a  straight  jacket  on  the  CMM. 

•  I  have  one 

•  Explains  overall  process,  but  each  project  must  devote  resources  to  interpret 
document,  and  train  the  team. 

•  A  hindrance  to  higher  levels.  Not  CMM-compliant.  Replace  with  generic  SDP  that 
addresses  process 

•  Most  of  it  should  make  us  more  inefficient. 

•  Incomplete.  Should  be  tied  completely  to  CMM. 

•  Why  do  we  not  follow  the  book  instead  of  making  up  our  own  version?  How  does 
CMM  fit  in? 

•  Probably  would  be  very  good  if  properly  managed  and  people  had  some  idea  of  how 
it  worked  and  it  could  be  explained  clearly  and  sensibly. 

•  It  required  a  lot  of  interpretation  and  translation  in  order  to  be  useful. 

•  Too  many  things  at  once.  Oversold  as  the  cure-all.  Didn’t  fit  well  with  production 
support  or  system  development. 

•  SMS  is  a  good  tool.  Proved  to  be  helpful  in  understanding  CMM.  However,  CMM  is 
what  rated  against.  Differences  in  interpreting  the  two  documents  caused  problems. 
CMM  is  what  we  must  master. 

•  It  is  totally  relevant  to  my  job. 

•  Usefulness  depends  on  implementation  within  the  project.  If  implementation  is  poor, 
SMS  is  seen  as  a  detriment. 

•  I  think  the  overali  process  is  a  sound  concept,  but  it  is  difficult  to  implement  within. 

•  SMS  covers  what  is  needed  to  define,  fix,  and  change  our  software.  It  does  seem 
wordy  but  covers  the  SMS 

•  If  all  facets  are  adhered  to,  and  the  customers,  who  do  not  fall  under  the  SMS 
umbrella,  understand  what  we  are  doing,  SMS  can  and  will  be  an  excellent  tool.  If 
not,  it  wiil  be  an  exercise  in  futility. 

•  Lacks  CMM  requirements  to  meet  Level  2. 

•  Needs  to  be  formatted  for  use  by  all  involved  and  contain  the  information  necessary 
to  implement,  i.e.,  checklists,  timelines,  process  flows,  etc. 

•  For  our  AIS,  we  are  instructed  to  also  follow  Military  Standard  498. 

•  Needs  to  be  interpreted  for  clarity. 

•  Found  it  to  be  very  beneficial  as  a  starting  and  reference  point  for  standardized 
procedures. 

•  Good  for  complex  changes,  complicated  and  too  involved  for  most  (shorter,  simpler) 
SCRs 

•  This  increase  paperwork.  Slow  down  processing,  increase  possible  mistakes 
because  of  added  steps.  Increase  uncertain  for  trying  to  learn  so  many  processes. 

•  Very  good  concept.  Implementation  time  is  not  sufficient  to  ensure  not  just 
compliance,  but  sufficient  knowledge  of  the  process  is  acquired  to  be  successfully 
repeatable 

•  There  is  no  documentation  on  how  to  use  the  process.  There  has  been  little  “buy  in” 
by  the  end  user  of  the  process. 

•  Format  useful  as  checklist  for  tasks/subtasks  to  perform.  Some  terminology  is 
difficult  to  understand. 

•  Doesn’t  have  CMM  stuff  in  it.  Evaluations  are  based  on  CMM.  SMS  won’t  get  you 
there. 

•  What  can  be  said  for  something  that  slows  the  process  to  its  crawl. 

•  Version  3  was  a  better  product  than  Version  4. 

•  It’s  awfully  big  and  cumbersome. 
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•  Too  many  options. 

•  The  DCPS  project  uses  the  high  level  steps  identified  in  the  SMS. 

•  Presentation  method  not  effective.  Some  very  good  concepts  and  practices 
contained  therein  but  needs  to  be  supplemented  with  graphics  and  other 
presentation  techniques  to  communicate  better. 

•  It  provided  good  procedural  assistance  where  we  were  short.  Initially,  the  format  was 
an  obstacle  to  usefulness.  Much  better  now. 

•  The  SMS  contains  all  the  processes  for  Level  2,  some  at  more  level  of  detail  than 
others. 

•  The  process  tends  to  bog  down  in  minute  details  that  sometimes  don’t  seem  to 
pertain  to  the  work  at  hand.  It  does,  however,  allow  for  exceptions  when  necessary. 

•  Too  hard  to  follow,  read,  implement,  and  not  realistic. 

•  Too  lengthy  to  use  effectively. 

16.  Please  list  any  additional  comments  on  Software  Process  Improvement: 

•  SPI  seems  to  be  an  academic  exercise  for  systems  phasing  out 

•  Not  convinced  that  SPI  is  worth  the  cost  in  time  and  resources 

•  Change  agents  and  sponsors  at  the  local  level  need  training  and  regular  periodic 
consultation  support 

•  The  SMS  needs  to  be  more  streamlined  and  graphical  with  templates 

•  The  SPI  program  process  (and  the  SEPG)  should  provide  more  samples  of  what  the 
procedures  and  plans  should  contain  early  on  (versus  after-the-fact  evaluations  by 
SEPG  and  SCE  teams  -  pointing  out  the  shortcomings). 

•  Develop  SPI  infrastructure  early  (PAL;  samples;  training  of  sponsors,  change  agents, 
and  development  staff;  streamline  status  reporting  method  and  format;  identify 
related  metrics  -  usable  for  site  and  HQ). 

•  Work  out  strategy  and  tactics  so  people  aren’t  overloaded  with  changes  (CMIS,  SPI, 
Metrics,  FPA,  etc.) 

•  Perform  pilots  to  work  out  kinks  in  the  process 

•  FPA  as  a  sizing  mechanism  is  not  cost  effective  for  already  established  AISs,  and 
does  not  help  in  comparing  estimated  to  actual  during  development 

•  Positive  aspects: 

•  documented  procedures 

•  SQA  processes  and  reviews 

•  justification  for  FFPs  and  schedules 

•  I  think  we  have  gained  some  bit  of  control  over  our  processes  -  however,  the  cost  for 
this  bit  of  improvement  has  been  great.  Increased  time  and  paperwork  have 
overshadowed  the  benefits  gained  and  to  be  gained  by  SPI.  Unfortunately,  many  of 
the  benefits  of  SPI  are  masked,  even  totally  hidden  under  the  amount  of  process  and 
documentation  required  to  reach  level. 

•  I  was  taught  and  believe  that  in  the  face  of  the  inevitable  consequences,  it  is  better  to 
say  nothing  than  complain. 

•  As  we  move  toward  Level  3,  emphasis  on  the  SPI  program  needs  to  be  renewed. 
Currently,  inactive  TWGs  need  to  be  reactivated  and/or  other  TWGs  started. 
SEPG/TWG  representatives  need  an  opportunity  to  rotate  out  of  the  duty  or  to  state 
an  interest  in  continuing  in  the  same  or  a  new  SPI  role. 

•  Must  address  cost  of  the  program  and  as  budgets  get  cut,  this  will  be  a  bigger  issue. 

•  SPI  is  a  worthwhile  initiative  but  sometimes  it  seems  to  take  more  time  than  the 
actual  work.  Perhaps  with  repetition,  the  extra  time  will  be  mitigated.  Currently,  it  is 
too  labor  intensive. 

•  There  is  a  great  value  in  implementing  disciplines  of  CMM,  however,  an  objective 
evaluation  of  Level  2  implementation  including  cost  effectiveness  should  be 
conducted  before  any  other  systems  begin.  This  is  especially  true  for  smaller 
systems  where  all  resources  are  tight. 
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•  With  current  and  future  budget  restraints  looming  on  the  horizon,  personnel 
maintaining  smaller  projects  feel  that  with  additional  SPI  requirements  levied  on  them 
may  mean  the  demise  of  other  projects.  An  overall  FSO  plan  needs  to  be  developed 
for  SPI  small  system  implementation  guidance. 

•  SPI  ideas  have  helped  our  organization  to  take  the  steps  needed  to  improve  our 
aged  software  development  activities  and  procedures.  DBAS  software  development 
procedures  and  activities  have  undergone  a  complete  renovation.  I  do  not  believe 
that  this  level  of  change  could  have  occurred  in  such  a  short  time  without  an  effort 
like  SPI. 

•  Most  of  the  objectives  of  SPI  we  were  already  doing  in  a  less  formal  manner  with  a 
record  of  very  few  production  problems.  SPI  has  added  elaborate  procedures  and 
formal  documentation  of  every  interim  minor  step  taken  for  the  sake  of  proof.  We 
have  just  as  many  production  problems  now  and  we  are  just  as  dependent,  if  not 
more  so,  on  expert  individuals.  SPI  was  also  forced  on  us  in  a  timeframe  which  did 
not  allow  institutionalizing  or  developing  more  automated  tools.  It  was  a  much  more 
painful  process  that  it  had  to  be.  In  summary,  there  is  a  tendency  for  SPI  to  become 
self-perpetuating  as  processes  become  more  complicated,  causing  more  human 
error  and  thus  requiring  more  oversight  and  more  procedures. 

•  Future  process  improvement  efforts  need  to  include  the  entire  organization  to 
succeed.  The  improvements  that  survived  the  push  for  the  grade  did  so  because  the 
projects  were  involved  in  their  development.  This  increased  the  buy-in  and 
acceptance  between  the  projects  have  a  stake  in  the  improvements.  Most  of  the 
changes  forced  downward  are  struggling.  Some  are  failing.  The  same  will  be  true  of 
future  efforts,  if  more  effort  is  not  made  to  involve  all  affected  parties.  History  is 
destined  to  repeat  itself. 

•  I  think  the  SCE  process  is  very  good.  The  team  had  .to  review  and  refine  vast 
amounts  of  data.  To  be  able  to  complete  this  task  in  such  a  short  time  is  really  an 
accomplishment.  I  thought  there  should  have  been  more  guidance  as  to  what  was 
passing  and  what  was  not  instead  of  relying  on  personal  opinions. 

•  The  need  for  local  Director  support  is  great.  The  need  for  training  in  many  support 
areas  is  also  great. 

•  My  belief  is  that  some  processes  should  be  put  in  place  to  help  improve  our  software 
development.  We  have  done  that  for  the  most  part.  However,  I  believe  that  we  need 
to  eliminate  some  “paper  shuffling"  to  allow  for  more  effective  time  management  in 
performing  software  development. 

•  Much  more  effort  needs  to  be  focused  on  fixing  current  process  problems. 
Practitioner  support  is  vital  to  successful  change,  and  many  practitioners  are  seeing 
too  little  improvement  for  too  much  work.  Note:  this  is  not  about  misplaced  focus. 
This  is  about  too  narrow  a  focus.  Both  executive/management  and  practitioner 
needs  need  to  be  met,  not  one  at  the  expense  of  the  other. 

•  I  believe  we’ll  all  be  better  off  if  we  can  get  everyone  to  ask  “How  can  I  help  to  make 
this  happen?"  instead  of  “Why  do  I  have  to  do  this?  a.k.a  “Not  another  initiative.” 

•  This  is  truly  a  great  improvement  and  step  in  the  right  direction  for  the  future. 

•  Too  many  “Chiefs”  doing  the  work  and  not  enough  “Indians’  involvement.” 

•  Really  need  FSA  director  and  division  chief  understanding.  Direction  and  support  to 
move  up  the  maturity  model.  It  makes  all  the  difference.  SPI  takes  time  and 
cooperation  plus  special  environment  to  take  hold. 

•  Software  shops,  like  SRD1 ,  should  not  be  required  to  do  PCA.  SRD1  does  not 
produce  any  hardware.  We  should  not  hesitate  to  use  the  term  “not  applicable”  when 
it  is  appropriate. 

•  We  need  more  emphasis  on  measuring  our  progress.  If  we  don’t  measure,  we  will 
never  know  whether  SPI/CMM  is  working  effectively  and  efficiently. 

•  Glad  to  see  effort  initiated,  but  need  more  open  communication  and  involvement 
across  the  board.  Worst  thing  we  could  do  is  let  someone  write  procedures  without 


137 


active  involvement  from  those  who  must  follow  them.  Standardization  and 
repeatability  great,  but  only  if  done  so  they  are  workable  and  efficient. 

•  SPI  program  managed  by  dates  instead  of  ensuring  processes  correct.  AISs  given 
deadlines  that  don’t  consider  many  difficulties,  risks,  uniqueness.  Hope  FSO/FSA 
management  would  focus  on  improving  processes  which  will  improve  products,  but 
they  continue  “I’d  rather  have  it  on  time”  philosophy. 

•  Process  was  like  blind  leading  blind  until  about  Oct  96. 

•  Making  slow  progress.  Seem  to  be  going  through  motions,  not  really  learning  how  to 
improve  project.  Much  more  cross  functional  involvement  needed.  No  sense 
improving  process  if  customer  not  brought  in  and  continues  to  expect  fire  drill 
responses. 

•  Continue  to  improve  -  to  be  the  best. 

•  I  am  very  uncomfortable  with  all  of  this.  I  will  do  as  ordered  or  requested  to  do. 

•  If  our  customers  see  this  system  as  a  money-consuming,  time-wasting  albatross, 
how  soon  will  it  be  before  we’re  sent  packing  and  they’re  hiring  an  outside  source  to 
do  the  same  job  we  could  have  done  for  them  but  cheaper  with  less  demand  for 
papenvork  and  absurd  activities. 

•  Must  happen 

•  Still  several  people  who  do  not  “buy  in”  to  SPI  program.  Important  for  those  involved 
in  SPI  to  not  criticize  or  bad-mouth  program.  More  positive  reinforcement  might  help. 

•  Just  beginning  program.  Only  morale  problems  so  far.  All  general  information  does 
is  make  people  feel  they’ve  got  a  lot  to  change.  Smaller  meetings  geared  to  specific 
processes  are  better.  Management  should  get  everyone  involved  earlier  in  the 
process.  People  need  to  be  better  informed. 

•  There  should  be  more  work  on  streamlining  the  SPI  process.  Perhaps  newer  tools  or 
better  ways  to  conduct  meetings  could  be  developed.  Since  language  and  spftware 
differ  for  each  mission,  it  is  hard  to  have  the  same  tools  everywhere,  but  more 
examples  might  help. 

•  If  I’ve  done  SPI  activities,  they’ve  not  been  designated  as  such,  so  I  don’t  know  when 
I’ve  done  them. 

•  It  obviously  is  being  done  by  some  people  because  I’ve  heard  of  it,  but  if  I’m  involved, 
or  it  trickles  down  to  my  level.  I’m  not  aware  of  it.  In  other  words,  I  do  what  I’m  told  to 
do,  but  nothing  has  ever  been  presented  as  “supporting  SPI” 

•  We  are  getting  a  lot  of  help  and  usefulness  out  of  SMS.  This  is  due  to  internal 
development  only,  not  based  on  help  or  guidance  from  outside  our  branch. 

•  More  specific  training  on  reviews,  audits,  follow-up,  writing  plans. 

•  It  appears  that  the  push/support  frorn  the  FSO  is  going  away.  I’m  not  sure  what  the 
value  added  is  to  going  to  Level  3,  however,  it  is  worth  getting  our  AIS  to  Level  2. 

•  The  jury’s  still  out. 

•  Level  of  documentation  should  be  reduced.  Cost  should  be  considered  to  ensure 
return  on  the  investment  in  a  relative  or  reasonable  amount  of  time. 

•  Need  more  training.  Need  more  involvement  from  FSO/SEPG.  Need  to  have  AISs 
that  have  reached  Level  2  give  their  documents/processes  to  FSO/SEPG  to  give  to 
other  AISs  trying  to  get  to  Level  2  instead  of  every  AIS  doing  the  same  thing  over  and 
over  and  over. 

•  One  problem  exists  and  will  continue  to  exist  at  all  levels  -  if  you  develop  a  new 
system  and  use  SPI  from  day  1,  you  will  build  a  great  AIS.  As  long  as  we  are 
required  to  keep  up  with  maintenance  and  new  development  on  current  AISs  to  the 
tune  of  a  more  than  full  work  week  for  our  limited  resources,  SPI  implementation  will 
take  too  long.  Please  understand  what  I  see  -  SPI  is  the  only  way  to  go.  Upper 
management  must  find  and  develop  a  better  way  for  us  to  get  there. 

•  Allow  systems  to  satisfy  the  CMM  without  using  SMS.  I  understand  most  of  the  FSAs 
did  not  use  the  SMS  for  their  methodology.  Why  is  FSAIN  different? 

•  Overall,  I  believe  there  has  been  a  high  level  of  success.  Everything  can  be 
improved.  This  process  indicates  an  organization’s  maturity. 
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In  order  to  get  SE  and  middle  managers  really  on  board,  they  need  to  have  a  say  in 
how  and  when.  Need  process  not  date  driven.  Need  recognition  for  positive 
behavior  (we  still  reward  heroes).  How  many  people  have  received 
awards/recognition  for  following  the  process  or  involvement  in  SPI.  Need  proper 
training.  Need  FSO  and  FSA  senior  management  support,  not  just  words  but  real 
action,  money,  compromise.  Management  needs  to  lead  and  set  the  example  (i.e., 
plan  and  track  those  plans).  No  crisis  management. 

There  is  a  subtle  danger  in  believing  a  particular  rating  is  the  end  of  the  story  and 
that  the  hard  work  of  improvement  is  behind  us.  A  good  example  is  the  fact  that  few 
of  the  projects  rated  Level  2  practice  configuration  management  with  the  rigor 
required  to  be  truly  effective. 

Continued  software  improvement  requires  corporate  level  policy  and  guidance. 
Although  work  has  been  done,  additional  work  must  be  completed  to  allow  FSO 
activities  to  advance  in  maturity. 

The  whole  SPI  effort,  more  than  anything  else,  helped  establish  a  unifying  sense  of 
being  across  the  FSAs  and  the  FSO  and  brought  us  in  common  business  and 
engineering  processes. 

Needs  to  be  managed  through  all  projects  with  a  corporate-wide  vision  versus 
individual  projects  or  sites. 

This  project  was  already  utilizing  the  basic  concepts  of  the  SMS.  The  processes 
were  repeatable  and  a  relatively  clean  product  was  released  to  users.  Utilizing  the 
SMS  required  our  group  to  generate  a  paper  trail  that  was  not  previously  in  place, 
and  hold  regular  formal  status  meetings.  Communication  between  levels  was 
improved. 

It  seems  to  me  that  FSAPE  should  have  SPI  documented  with  all  its  “standards”  for 
system  development/maintenance  and  each  application  system  should  only  need 
unique  or  addendum  documentation  requirements. 

SPI  is  very  important  and  can  lead  to  many  benefits  in  areas  of  cost  savings  and 
customer  satisfaction.  It  must  be  treated  seriously  and  funded/staffed  at  an 
appropriate  level  to  make  it  effective  at  each  FSA. 


140 


APPENDIX  C.  INTERVIEW  QUESTIONS 


Initial  Interview  Questions  for  FSA-KC 

•  Why  is  SPI  important  to  FS  A? 

•  What  incentives  are  there  for  you  to  support  SPI? 

•  How  can  the  SEPG  help  you? 

•  What  are  the  anticipated  hardspots  in  achieving  CMM  level  3? 

•  What  incentives  are  there  for  you  to  adopt  SPI  initiatives? 

•  Are  there  external  factors  that  enhance  or  detract  from  the  effectiveness  of  SPI 
efforts? 

Interview  Questions  for  Director,  FSA 

•  What  is  the  mission  of  FSA? 

•  What  can  software  process  improvement  do  for  FSA? 

•  What  incentives  are  there  for  key  management  to  champion  SPI? 

•  What  is  the  mission  of  the  SEPG? 

•  What  are  the  anticipated  hardspots  in  achieving  CMM  level  3? 

Follow-up  Interview  Questions  for  FSA-KC 

•  What  concrete  observable  outcomes  will  result  from  the  current  process  improvement 
efforts? 

•  Do  the  proposed  changes  make  the  employees’  job  easier  or  harder? 

•  Are  the  changes  technically  familiar  to  members  of  the  organization? 

•  Was  input  frorn  the  section/branch  personnel  included  in  the  formulation  of  the 
process  improvement  plan? 

•  What  specific  parts  of  the  organization  provide  the  most  opportunity  for  increases  in 
efficiency  and  quality? 
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•  What  systems  need  improving,  (example  technological  systems,  management 
systems,  management  practices,  or  organizational  structures)? 

•  The  most  important  thing  to  the  leadership  of  FSA  is.. 

•  Whats  does  the  director  (FSA)  say  about  SPI?  What  does  he  do? 

•  What  does  the  center  director  say  about  SPI?  What  does  she  do? 

Interview  questions  for  Customer  Representatives 

•  What  are  your  initial  impressions  on  software  process  improvement? 

•  Have  you  been  briefed  on  SPI? 

•  Will  SPI  initiatives  help  you?  In  what  way?  Are  you  willing  to  pay  for  SPI  efforts? 
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